Parallel cpp functions in R? Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern) Data science time! April 2019 and salary with experience The Ask Question Wizard is Live!Using Rcpp functions inside of R's par*apply functions from the parallel packageWhat is the difference between concurrency and parallelism?What is the difference between concurrent programming and parallel programming?Grouping functions (tapply, by, aggregate) and the *apply familyShould I always use a parallel stream when possible?Custom thread pool in Java 8 parallel streamR Parallel - connecting to remote coresHow do I share C++ functions in Rcpp-based libraries between R packages?Why does foreach %dopar% get slower with each additional node?Parallel processing and querying SQL with dplyr or pool: MySQL server has gone awayParallelized estimation of glarma time series models in R with parLapply instable
Vertical ranges of Column Plots in 12
First paper to introduce the "principal-agent problem"
Any stored/leased 737s that could substitute for grounded MAXs?
Google .dev domain strangely redirects to https
Is this Kuo-toa homebrew race balanced?
As a dual citizen, my US passport will expire one day after traveling to the US. Will this work?
Is Mordenkainens' Sword under powered?
Why can't fire hurt Daenerys but it did to Jon Snow in season 1?
NIntegrate on a solution of a matrix ODE
How do Java 8 default methods hеlp with lambdas?
Short story about astronauts fertilizing soil with their own bodies
Why complex landing gears are used instead of simple, reliable and light weight muscle wire or shape memory alloys?
Was the pager message from Nick Fury to Captain Marvel unnecessary?
How do you write "wild blueberries flavored"?
Does a random sequence of vectors span a Hilbert space?
Problem with display of presentation
In musical terms, what properties are varied by the human voice to produce different words / syllables?
Did John Wesley plagiarize Matthew Henry...?
Where and when has Thucydides been studied?
How to resize main filesystem
2018 MacBook Pro won't let me install macOS High Sierra 10.13 from USB installer
How to make triangles with rounded sides and corners? (squircle with 3 sides)
Twin's vs. Twins'
Why did Bronn offer to be Tyrion Lannister's champion in trial by combat?
Parallel cpp functions in R?
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern)
Data science time! April 2019 and salary with experience
The Ask Question Wizard is Live!Using Rcpp functions inside of R's par*apply functions from the parallel packageWhat is the difference between concurrency and parallelism?What is the difference between concurrent programming and parallel programming?Grouping functions (tapply, by, aggregate) and the *apply familyShould I always use a parallel stream when possible?Custom thread pool in Java 8 parallel streamR Parallel - connecting to remote coresHow do I share C++ functions in Rcpp-based libraries between R packages?Why does foreach %dopar% get slower with each additional node?Parallel processing and querying SQL with dplyr or pool: MySQL server has gone awayParallelized estimation of glarma time series models in R with parLapply instable
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
So I have 2 custom Rcpp functions CustFunc1(x,y) and CustFunc2(a,b). Both are computationally demanding (thus the c++). My question is there a way in R to run the concurrently with the dopar package? Or can I do a system call to directly access the cpp code and try to do parallel via some command line tools?
Right now the flow is:
result1=CustFunc(x,y) ##Takes 20 minutes
result2=CustFunc(a,b) ## Take 20 minutes
get_both <- function(x)
foreach(i = seq_along(x)) %dopar%
result1=CustFunc(x,y)
result2=CustFunc(a,b)
get_both$result1 == result1 #???
get_both$result2 == result2 ##??
r parallel-processing doparallel
asked Mar 9 at 0:32
K.J.J.KK.J.J.K
12316
add a comment |
So I have 2 custom Rcpp functions CustFunc1(x,y) and CustFunc2(a,b). Both are computationally demanding (thus the c++). My question is there a way in R to run the concurrently with the dopar package? Or can I do a system call to directly access the cpp code and try to do parallel via some command line tools?
Right now the flow is:
result1=CustFunc(x,y) ##Takes 20 minutes
result2=CustFunc(a,b) ## Take 20 minutes
get_both <- function(x)
foreach(i = seq_along(x)) %dopar%
result1=CustFunc(x,y)
result2=CustFunc(a,b)
get_both$result1 == result1 #???
get_both$result2 == result2 ##??
r parallel-processing doparallel
asked Mar 9 at 0:32
K.J.J.KK.J.J.K
12316
1
The best thing to do is roll the functions into a package, then you should be able to use the functions in parallel no problem. See, for example, this answer to a related question on Stack Overflow, or this blog post
– duckmayr
Mar 9 at 2:23
add a comment |
So I have 2 custom Rcpp functions CustFunc1(x,y) and CustFunc2(a,b). Both are computationally demanding (thus the c++). My question is there a way in R to run the concurrently with the dopar package? Or can I do a system call to directly access the cpp code and try to do parallel via some command line tools?
Right now the flow is:
result1=CustFunc(x,y) ##Takes 20 minutes
result2=CustFunc(a,b) ## Take 20 minutes
get_both <- function(x)
foreach(i = seq_along(x)) %dopar%
result1=CustFunc(x,y)
result2=CustFunc(a,b)
get_both$result1 == result1 #???
get_both$result2 == result2 ##??
r parallel-processing doparallel
asked Mar 9 at 0:32
K.J.J.KK.J.J.K
12316
So I have 2 custom Rcpp functions CustFunc1(x,y) and CustFunc2(a,b). Both are computationally demanding (thus the c++). My question is there a way in R to run the concurrently with the dopar package? Or can I do a system call to directly access the cpp code and try to do parallel via some command line tools?
Right now the flow is:
result1=CustFunc(x,y) ##Takes 20 minutes
result2=CustFunc(a,b) ## Take 20 minutes
get_both <- function(x)
foreach(i = seq_along(x)) %dopar%
result1=CustFunc(x,y)
result2=CustFunc(a,b)
get_both$result1 == result1 #???
get_both$result2 == result2 ##??
r parallel-processing doparallel
r parallel-processing doparallel
asked Mar 9 at 0:32
K.J.J.KK.J.J.K
12316
asked Mar 9 at 0:32
K.J.J.KK.J.J.K
12316
edited Mar 9 at 1:53
K.J.J.K
asked Mar 9 at 0:32
K.J.J.KK.J.J.K
12316
asked Mar 9 at 0:32
K.J.J.KK.J.J.K
12316
asked Mar 9 at 0:32
K.J.J.KK.J.J.K
12316
12316
1
The best thing to do is roll the functions into a package, then you should be able to use the functions in parallel no problem. See, for example, this answer to a related question on Stack Overflow, or this blog post
– duckmayr
Mar 9 at 2:23
add a comment |
1
The best thing to do is roll the functions into a package, then you should be able to use the functions in parallel no problem. See, for example, this answer to a related question on Stack Overflow, or this blog post
– duckmayr
Mar 9 at 2:23
1
1
The best thing to do is roll the functions into a package, then you should be able to use the functions in parallel no problem. See, for example, this answer to a related question on Stack Overflow, or this blog post
– duckmayr
Mar 9 at 2:23
The best thing to do is roll the functions into a package, then you should be able to use the functions in parallel no problem. See, for example, this answer to a related question on Stack Overflow, or this blog post
– duckmayr
Mar 9 at 2:23
add a comment |
1 Answer
1
active
oldest
votes
If your functions take on the order of 20 min I would really consider rewriting them using the RcppPaallel package, it isn’t too much harder than straight cpp. That way you can use more threads, but run your functions one at a time. When I did this with my own code, I cut my computation time down from about 15 min to less than 2.
answered Mar 10 at 3:36
skatzskatz
295
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55072826%2fparallel-cpp-functions-in-r%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
If your functions take on the order of 20 min I would really consider rewriting them using the RcppPaallel package, it isn’t too much harder than straight cpp. That way you can use more threads, but run your functions one at a time. When I did this with my own code, I cut my computation time down from about 15 min to less than 2.
answered Mar 10 at 3:36
skatzskatz
295
add a comment |
If your functions take on the order of 20 min I would really consider rewriting them using the RcppPaallel package, it isn’t too much harder than straight cpp. That way you can use more threads, but run your functions one at a time. When I did this with my own code, I cut my computation time down from about 15 min to less than 2.
answered Mar 10 at 3:36
skatzskatz
295
add a comment |
If your functions take on the order of 20 min I would really consider rewriting them using the RcppPaallel package, it isn’t too much harder than straight cpp. That way you can use more threads, but run your functions one at a time. When I did this with my own code, I cut my computation time down from about 15 min to less than 2.
answered Mar 10 at 3:36
skatzskatz
295
If your functions take on the order of 20 min I would really consider rewriting them using the RcppPaallel package, it isn’t too much harder than straight cpp. That way you can use more threads, but run your functions one at a time. When I did this with my own code, I cut my computation time down from about 15 min to less than 2.
answered Mar 10 at 3:36
skatzskatz
295
answered Mar 10 at 3:36
skatzskatz
295
answered Mar 10 at 3:36
skatzskatz
295
answered Mar 10 at 3:36
skatzskatz
295
295
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55072826%2fparallel-cpp-functions-in-r%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
The best thing to do is roll the functions into a package, then you should be able to use the functions in parallel no problem. See, for example, this answer to a related question on Stack Overflow, or this blog post
– duckmayr
Mar 9 at 2:23