Perfomance Java 8 lambdas

classic Classic list List threaded Threaded
9 messages Options
pgarg pgarg
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Perfomance Java 8 lambdas

asked by vladimir

I'm running CreditRiskExample on 11 nodes cluster (16 Gb RAM, core i7, 1 Tb each node). I'm using parallel stream of Java 8. Why parralel version which uses Java 8 lambdas (Code B) is slower then not parallel version (Code A)?

Code A (Credit risk [crdRisk=627147.908398507, duration=73667ms]):

double[] losses = new double[num];
for (int i = 0; i < num; i++)
{
    for (Credit crd : portfolio)
    {
        int remDays = Math.min(crd.getRemainingTerm(), horizon);

        if (rndGen.nextDouble() >= 1 - crd.getDefaultProbability(remDays))
            losses[i] += (1 + crd.getAnnualRate() * Math.min(horizon, crd.getRemainingTerm()) / 365)
                * crd.getRemainingAmount();
        else
            losses[i] -= crd.getAnnualRate() * Math.min(horizon, crd.getRemainingTerm()) / 365 *
                crd.getRemainingAmount();
    }
}
------------------------------------------------------------------------------
Code B (Credit risk [crdRisk=752244.1112362606, duration=151260ms]):

double[] losses = new double[num];

IntStream.range(0, num).parallel().forEach(i ->
{
    for (Credit crd : portfolio)
    {
        int remDays = Math.min(crd.getRemainingTerm(), horizon);

        if (rndGen.nextDouble() >= 1 - crd.getDefaultProbability(remDays))
            losses[i] += (1 + crd.getAnnualRate() * Math.min(horizon, crd.getRemainingTerm()) / 365)
                * crd.getRemainingAmount();
        else
            losses[i] -= crd.getAnnualRate() * Math.min(horizon, crd.getRemainingTerm()) / 365 *
                crd.getRemainingAmount();
    }
}
);

-----
This post is migrated from now discontinued Apache Ignite forum at
http://apacheignite.readme.io/v1.0/discuss
pgarg pgarg
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Perfomance Java 8 lambdas

commented by vladimir

I'm used this parameters:

Credit[] portfolio = new Credit[5000]; Random rnd = new Random();

// Generate some test portfolio items.
for (int i = 0; i < portfolio.length; i++) {
    portfolio[i] = new Credit(
        50000 * rnd.nextDouble(), // Credit amount.
        rnd.nextInt(1000), // Credit term in days.
        rnd.nextDouble() / 10, // APR.
        rnd.nextDouble() / 20 + 0.02 // EDF.
    );
}

// Forecast horizon in days.
int horizon = 365;

// Number of Monte-Carlo iterations.
int iter = 1000000;

// Percentile.
double percentile = 0.95;

-----
This post is migrated from now discontinued Apache Ignite forum at
http://apacheignite.readme.io/v1.0/discuss
pgarg pgarg
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Perfomance Java 8 lambdas

commented by dmitriy setrakyan

I can see 2 possible reasons for this:

Ignite already runs all jobs in parallel and utilizes CPU cores to the maximum. In this case, adding more concurrency will only slow you down, because tasks start competing for CPU cores against each other.
The list may not be large enough to get advantage of the parallel processing.
I am tempted to think that the slow down is caused by the first (1) point.

-----
This post is migrated from now discontinued Apache Ignite forum at
http://apacheignite.readme.io/v1.0/discuss

pgarg pgarg
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Perfomance Java 8 lambdas

commented by dmitriy setrakyan

I think what you can do is check in Task Monitor (Win) or Activity Monitor (Mac) if all the cores are utilized by Ignite jobs during the execution. If so, then adding additional threads to execute the list in parallel will only slow you down.

-----
This post is migrated from now discontinued Apache Ignite forum at
http://apacheignite.readme.io/v1.0/discuss

pgarg pgarg
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Perfomance Java 8 lambdas

commented by vladimir

I looked at the CPU usage. When I use Code A all nodes uses only one core. And duration of all task is ~ 73667ms.

When I use Code B (which uses Java 8 lambdas) all nodes uses all cores. And duration of all task is ~ 151260ms.

very strange...

I can send screenshots.

-----
This post is migrated from now discontinued Apache Ignite forum at
http://apacheignite.readme.io/v1.0/discuss

pgarg pgarg
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Perfomance Java 8 lambdas

commented by vladimir

I still don't recieve an answer

-----
This post is migrated from now discontinued Apache Ignite forum at
http://apacheignite.readme.io/v1.0/discuss

pgarg pgarg
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Perfomance Java 8 lambdas

commented by dmitriy setrakyan

As I suggested before, I believe Ignite already utilizes all the Cores, so if you try to do something additional in parallel, you are essentially beginning to compete with Ignite jobs.

To check that, look at your Core utilization when you are NOT using parallel streams and see if all the cores are utilized by Ignite. If yes, then parallel streams will only slow it down because they will add thread contention.

-----
This post is migrated from now discontinued Apache Ignite forum at
http://apacheignite.readme.io/v1.0/discuss

pgarg pgarg
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Perfomance Java 8 lambdas

commented by vladimir

When I'm NOT using parallel streams only one core utilized by Ignite. When I'm using parallel streams all the cores are utilized by Ignite.

-----
This post is migrated from now discontinued Apache Ignite forum at
http://apacheignite.readme.io/v1.0/discuss

pgarg pgarg
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Perfomance Java 8 lambdas

commented by dmitriy setrakyan

Let me try to reproduce. Will post my findings here.

-----
This post is migrated from now discontinued Apache Ignite forum at
http://apacheignite.readme.io/v1.0/discuss

Loading...