Executor Framework
Executor Framework
Creating a Thread for every unit of work is straightforward at small scale, but it does not hold up once real traffic arrives — each thread costs real memory and OS scheduling overhead, with nothing capping how many can exist at once. The Executor Framework, part of java.util.concurrent since Java 5, separates what runs from how many threads run it, managing a pool of reusable worker threads behind a simple submit() call.
What Is an Executor?
ExecutorService is the interface that decouples submitting work from running it — code calls submit() or execute() with a task, and the pool decides which worker thread actually runs it and when, based on how many threads exist and how busy they are.
Why Thread Safety Matters Here
Every raw Thread created directly costs real memory for its stack and real OS scheduling overhead, and nothing about new Thread(...) caps how many can exist at once. A service that spins up a thread per incoming request looks perfectly fine in a demo with five requests, then falls over in production the moment traffic actually spikes, since the JVM will keep creating threads until the operating system itself refuses. Left uncoordinated, this is not just a performance problem — a spike in unbounded thread creation is a common root cause of an outage that looks unrelated to the code that actually triggered it.
A pool does not just save typing over manual thread management — it puts a hard ceiling on concurrency that no individual call site has to remember to enforce.
How It Works
One sentence before the diagram: a fixed-size pool queues work beyond its thread count instead of creating a new thread for it.
Submitted tasks Fixed pool (3 worker threads)
task 0 ---+
task 1 ---+
task 2 ---+--> [ task queue ] --> Worker 1 --\
task 3 ---+ Worker 2 ---+--> results written back
task 4 ---+ Worker 3 --/
(waits here until a worker becomes free)
Every task beyond the pool's thread count sits in the queue until a worker finishes its current task and picks up the next one — the pool's size, not the number of submitted tasks, is what determines how much actually runs at once.
Managing raw threads directly means creating, starting, and joining every one of them by hand, with no limit on how many exist at once.
1// File: BeforeExecutor.java
2
3public class BeforeExecutor {
4 public static void main(String[] args) throws InterruptedException {
5 String[] results = new String[5];
6 Thread[] threads = new Thread[5];
7
8 for (int i = 0; i < 5; i++) {
9 int taskId = i;
10 threads[i] = new Thread(() -> {
11 results[taskId] = "Resized image " + taskId;
12 });
13 }
14
15 for (Thread t : threads) {
16 t.start();
17 }
18 for (Thread t : threads) {
19 t.join();
20 }
21
22 for (String result : results) {
23 System.out.println(result);
24 }
25 }
26}Output:
Resized image 0
Resized image 1
Resized image 2
Resized image 3
Resized image 4
ExecutorService replaces the manual create/start/join bookkeeping with a pool that caps how many threads actually exist, regardless of how many tasks are submitted.
1// File: AfterExecutor.java
2import java.util.concurrent.*;
3
4public class AfterExecutor {
5 public static void main(String[] args) throws InterruptedException {
6 String[] results = new String[5];
7
8 try (ExecutorService executor = Executors.newFixedThreadPool(3)) {
9 for (int i = 0; i < 5; i++) {
10 int taskId = i;
11 executor.submit(() -> {
12 results[taskId] = "Resized image " + taskId;
13 });
14 }
15 }
16
17 for (String result : results) {
18 System.out.println(result);
19 }
20 }
21}Output:
Resized image 0
Resized image 1
Resized image 2
Resized image 3
Resized image 4
Five tasks ran here on a pool of only three threads — the pool queues work beyond its thread count automatically, something the manual version has no equivalent for at all. Each task writes to its own index of the results array, so there is no race between them even without explicit synchronization, and the try-with-resources block's close() waits for every submitted task to finish before the results are read.
Code Examples
submit() returns a Future, which can be used to retrieve a computed value once the task completes.
1// File: ExecutorFutureExample.java
2import java.util.concurrent.*;
3
4public class ExecutorFutureExample {
5 public static void main(String[] args) throws Exception {
6 try (ExecutorService executor = Executors.newSingleThreadExecutor()) {
7 Future<Integer> future = executor.submit(() -> 10 * 10);
8 System.out.println("Result: " + future.get());
9 }
10 }
11}Output:
Result: 100
Before Java 19's close(), the standard shutdown sequence was shutdown() — which stops accepting new tasks but lets already-submitted ones finish — followed by awaitTermination() to block until they actually do.
1// File: ShutdownAwaitTerminationExample.java
2import java.util.concurrent.*;
3
4public class ShutdownAwaitTerminationExample {
5 public static void main(String[] args) throws InterruptedException {
6 ExecutorService executor = Executors.newFixedThreadPool(2);
7 int[] results = new int[2];
8
9 executor.submit(() -> { results[0] = 10 * 10; });
10 executor.submit(() -> { results[1] = 20 * 20; });
11
12 executor.shutdown();
13 executor.awaitTermination(5, TimeUnit.SECONDS);
14
15 System.out.println("Results: " + results[0] + ", " + results[1]);
16 }
17}Output:
Results: 100, 400
Each task writes to its own array index, so no synchronization is needed between the two — awaitTermination() blocking until both finish is what makes reading the array afterward safe. Beyond parallelizing independent batch work, an ExecutorService is also the standard tool for limiting concurrency deliberately with newFixedThreadPool(n), and for background task processing with newSingleThreadExecutor() to serialize work off the calling thread without manual thread management.
Real-World Example
A background service resizes a batch of uploaded images into thumbnails, submitting each resize as its own task to a fixed-size pool rather than spawning a thread per image.
1// File: ThumbnailTask.java
2
3public record ThumbnailTask(String imageName, int width, int height) {}1// File: ThumbnailService.java
2import java.util.concurrent.*;
3import java.util.*;
4
5public class ThumbnailService {
6
7 public List<String> resizeAll(List<ThumbnailTask> tasks) throws InterruptedException {
8 String[] results = new String[tasks.size()];
9
10 try (ExecutorService executor = Executors.newFixedThreadPool(4)) {
11 for (int i = 0; i < tasks.size(); i++) {
12 int index = i;
13 ThumbnailTask task = tasks.get(i);
14 executor.submit(() -> {
15 results[index] = task.imageName() + " -> " + task.width() + "x" + task.height();
16 });
17 }
18 }
19
20 return List.of(results);
21 }
22}1// File: ThumbnailServiceDemo.java
2import java.util.*;
3
4public class ThumbnailServiceDemo {
5 public static void main(String[] args) throws InterruptedException {
6 List<ThumbnailTask> tasks = List.of(
7 new ThumbnailTask("sunset.jpg", 150, 100),
8 new ThumbnailTask("portrait.png", 120, 160),
9 new ThumbnailTask("banner.jpg", 300, 80)
10 );
11
12 ThumbnailService service = new ThumbnailService();
13 List<String> results = service.resizeAll(tasks);
14
15 results.forEach(System.out::println);
16 }
17}Output:
sunset.jpg -> 150x100
portrait.png -> 120x160
banner.jpg -> 300x80
A mistake that appears often in fresher pull requests is creating a brand new Thread for every incoming task, which works fine at low volume but exhausts OS resources once real traffic arrives. A fixed-size pool caps how many threads exist at once regardless of how many tasks are submitted, exactly as ThumbnailService does here with newFixedThreadPool(4) handling however many images arrive without spawning a thread per image. CompletableFuture, covered next in this section, builds on this same idea of asynchronous work but adds fluent chaining on top of what a raw Future can do, and ForkJoinPool, covered later, is itself an ExecutorService specialized for divide-and-conquer recursive work.
Best Practices
Prefer a bounded pool, such as newFixedThreadPool(n), over an unbounded newCachedThreadPool() when the workload's concurrency needs to stay predictable and resource usage bounded.
Always shut down an ExecutorService — through try-with-resources and close(), or through shutdown() followed by awaitTermination() — rather than leaving it running indefinitely.
Avoid creating a new executor for every request or task; create one, reuse it for the workload's lifetime, and shut it down when that workload is done.
Size a pool based on the nature of the work — roughly the number of CPU cores for CPU-bound tasks, and often higher for I/O-bound tasks that spend most of their time waiting.
Common Mistakes
Never shutting down an ExecutorService at all leaves its threads running — they are non-daemon by default, which keeps the JVM alive even after main() itself has returned.
1// Illustrative only - do not rely on this pattern: without shutdown(),
2// the pool's threads are non-daemon and keep the JVM running
3// even though main() itself has already returned
4ExecutorService executor = Executors.newFixedThreadPool(2);
5executor.submit(() -> System.out.println("task done"));
6// missing: executor.shutdown() or a try-with-resources blockAssuming an exception thrown inside a task submitted via submit() surfaces immediately overlooks that it is captured inside the returned Future instead, and only reappears once get() is actually called.
1// File: SwallowedExceptionMistake.java
2import java.util.concurrent.*;
3
4public class SwallowedExceptionMistake {
5 public static void main(String[] args) {
6 try (ExecutorService executor = Executors.newSingleThreadExecutor()) {
7 Future<Integer> future = executor.submit(() -> {
8 throw new ArithmeticException("divide by zero");
9 });
10
11 try {
12 future.get();
13 } catch (ExecutionException e) {
14 System.out.println("Caught via Future: " + e.getCause().getClass().getSimpleName());
15 } catch (InterruptedException e) {
16 Thread.currentThread().interrupt();
17 }
18 }
19 }
20}Output:
Caught via Future: ArithmeticException
If future.get() were never called here, this ArithmeticException would vanish without a trace — no stack trace printed, no log entry, nothing. execute(), by contrast, reports an uncaught task exception to the thread's default uncaught exception handler immediately, since there is no Future to hold onto it.
Interview Questions
Q1. Why is manually creating a new Thread for every task considered poor practice at scale?
Each thread costs real memory for its stack and real OS scheduling overhead, with nothing capping how many exist at once — under real traffic, this exhausts system resources. A pooled ExecutorService caps the number of threads regardless of how many tasks are submitted. The nuance interviewers are listening for is whether you can name the actual resource cost — stack memory and OS scheduling — rather than just saying threads are "expensive" without explaining why.
Q2. What is the difference between execute() and submit() on an ExecutorService?
execute(Runnable) runs a task with no way to retrieve a result or a thrown exception directly — an uncaught exception goes to the thread's default handler. submit() returns a Future, letting the caller retrieve a result or an exception via get(), as demonstrated in this article's Common Mistakes section. Interviewers are often checking whether you know exceptions from submit()-ed tasks are silently swallowed unless get() is actually called.
Q3. What is the difference between shutdown() and shutdownNow()?
shutdown() stops accepting new tasks but lets already-submitted ones run to completion. shutdownNow() attempts to stop actively executing tasks by interrupting them and returns a list of tasks that were queued but never started. The nuance here is that shutdownNow() only attempts interruption — a task ignoring InterruptedException can keep running regardless.
Q4. What happens to an ExecutorService's threads if shutdown() is never called?
They keep running indefinitely, since a pool's threads are non-daemon by default — this can keep the JVM alive even after main() has returned, exactly as illustrated in this article's Common Mistakes section. This is a favorite service-company recall question precisely because the failure mode is so easy to reproduce and so surprising to someone who has not seen it before.
Q5. What is the difference between newFixedThreadPool() and newCachedThreadPool()?
newFixedThreadPool(n) maintains exactly n threads, queuing any work beyond that. newCachedThreadPool() creates new threads as needed with no upper bound, reusing idle ones — convenient for short-lived bursty tasks, but risky for sustained high load since nothing caps the thread count. Product-company interviewers tend to push further here, asking you to justify which one you would pick for a specific workload and why.
Q6. What does close() do on an ExecutorService, and which Java version introduced it?
Introduced in Java 19, close() initiates an orderly shutdown and blocks until all tasks complete, making an ExecutorService usable directly in a try-with-resources block — exactly as every example in this article does. Knowing the exact version signals you keep up with the platform rather than only knowing the pre-19 shutdown()/awaitTermination() pattern.
Q7. What happens to an exception thrown inside a task submitted via submit(), if get() is never called on the returned Future?
It is silently discarded — the exception is stored inside the Future but never surfaces anywhere unless get() is called on that specific Future, as demonstrated in this article's SwallowedExceptionMistake example. This is precisely the nuance a product-company interviewer is probing for: do you understand this is a genuine production risk, not just a theoretical corner case.
FAQs
Is it safe to reuse the same ExecutorService across many batches of work?
Yes, and it is the recommended pattern — creating one executor and submitting many batches of tasks to it over its lifetime is far more efficient than creating a new executor per batch.
Does calling shutdown() immediately stop all running tasks?
No. shutdown() only stops new tasks from being accepted — tasks already running or queued continue to completion. shutdownNow() is the method that attempts to stop currently running tasks via interruption.
What is ForkJoinPool.commonPool(), and is it an ExecutorService?
It is a shared, JVM-wide ForkJoinPool instance used internally by parallel streams and by CompletableFuture's default async methods. Yes, it is an ExecutorService, since ForkJoinPool extends AbstractExecutorService — covered in this section's dedicated ForkJoinPool article.
Can an ExecutorService be used with virtual threads?
Yes, Executors.newVirtualThreadPerTaskExecutor() returns an ExecutorService that starts a new virtual thread per submitted task — covered in this series' Virtual Threads article.
What happens if more tasks are submitted than a fixed pool's thread count?
The extra tasks wait in an internal queue until a thread becomes free, exactly as five tasks did on a pool of three threads in this article's first example.
Does submit() block the calling thread until the task completes?
No. submit() returns immediately with a Future; the task runs asynchronously on the pool. Only calling future.get() blocks, and only until that specific task finishes.
How many threads should a fixed thread pool typically be sized to?
For CPU-bound work, roughly the number of available CPU cores is a reasonable starting point, since more threads than cores mostly just adds contention. For I/O-bound work, a higher count is often appropriate, since threads spend much of their time waiting rather than actively computing.
Summary
The Executor Framework replaces manual thread creation with a managed pool, capping how many threads actually exist regardless of how many tasks are submitted, and returning a Future for retrieving a result or a captured exception later. close(), added in Java 19, makes shutting one down as simple as any other try-with-resources resource.
The habit worth carrying forward from this article's thumbnail service example is sizing a pool deliberately for the kind of work it handles, and always ensuring an executor is shut down — whether through close() or the older shutdown()/awaitTermination() pair — rather than leaving its threads running indefinitely.
What to Read Next
Learn how to run tasks asynchronously and combine their results.