C# · Course navigation

C#: Parallel.For with Interlocked

Spread loop iterations over cores and combine results safely with an atomic add.

On this page

Before you start

C# lambdas and the threads lesson.

Spread loop iterations over cores and combine results safely with an atomic add.
Predict the sample output and explain the state changes.
Complete the practice task and check the boundary cases.

The concept

Parallel.For runs the iterations of a loop on several threads. The body must be safe to run concurrently: iterations may not depend on each other, and shared variables must be protected. Adding to one shared total from every iteration is a data race if done with a plain +=.

Interlocked.Add performs the addition as one atomic operation, so no update is lost. PLINQ is a higher-level alternative: AsParallel().Sum combines partial sums for you. Both give the same exact answer here because integer addition is associative, so the order of the additions does not matter.

Code example

using System;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;

class Program {
  static void Main() {
    long total = 0;
    Parallel.For(1, 1001, i => {
      Interlocked.Add(ref total, (long)i * i);
    });
    Console.WriteLine("Parallel.For: " + total);

    long viaLinq = Enumerable.Range(1, 1000).AsParallel().Sum(i => (long)i * i);
    Console.WriteLine("PLINQ:        " + viaLinq);
  }
}

Output

Parallel.For: 333833500
PLINQ:        333833500

Example details

Follow the code

Open the walkthrough to inspect the example one step at a time.

Show guided walkthrough

Guided walkthroughReady

1using System;

2using System.Linq;

3using System.Threading;

4using System.Threading.Tasks;

6class Program {

7 static void Main() {

8 long total = 0;

9 Parallel.For(1, 1001, i => {

10 Interlocked.Add(ref total, (long)i * i);

11 });

12 Console.WriteLine("Parallel.For: " + total);

14 long viaLinq = Enumerable.Range(1, 1000).AsParallel().Sum(i => (long)i * i);

15 Console.WriteLine("PLINQ: " + viaLinq);

16 }

17}

Follow the example

Start to inspect the source, values, and output. These prepared steps explain the published sample; grouped loop steps describe multiple iterations.

View the complete dry-run table

Prepared trace of the published program. Some rows group repeated iterations.
What happens	State after step	Output so far
A shared total updated from many threads.	`total = 0`	—
Each iteration adds its square atomically, so no addition is lost.	`total grows by i² per iteration`	—
The loop returns only when every iteration is done.	`total = 333833500`	Parallel.For: 333833500
PLINQ computes the same sum with partial sums per thread.	`total = 333833500`	Parallel.For: 333833500 PLINQ: 333833500

Behind the scenes

How C# processes this code

C# compiles to code for the .NET runtime, which invokes the program entry point.

Cost and common mistakes

O(n) work with ideal elapsed time O(n / p).

total += i * i inside Parallel.For is a data race and usually gives a smaller, changing result.
Contending on one atomic variable limits the speed-up. Thread-local partial sums combined at the end scale better, which is what PLINQ does.
Floating-point addition is not associative, so parallel sums of doubles can differ slightly from a serial sum.

Try it yourself

What is the sum of squares from 1 to 10?

Reveal answer and reasoning

385, from 10 · 11 · 21 / 6.