What is nn.Sequential?

In PyTorch, nn.Sequential is a container (a simple model builder) that allows you to stack layers in order, one after another. Each layer’s output automatically becomes the next layer’s input.

import torch
import torch.nn as nn

model = nn.Sequential(
    nn.Linear(10, 20),   # Layer 1: Linear transformation (input 10 → output 20)
    nn.ReLU(),           # Layer 2: ReLU activation
    nn.Linear(20, 1)     # Layer 3: Linear transformation (input 20 → output 1)
)
                

Conceptually

Think of nn.Sequential like a pipeline:

Input → Layer1 → Layer2 → Layer3 → Output

When Should You Use It?

1. Simplicity

If your model is a simple chain of layers (e.g., Linear → ReLU → Linear → Sigmoid), nn.Sequential makes it clean, short, and readable.

model = nn.Sequential(
    nn.Linear(10, 20),
    nn.ReLU(),
    nn.Linear(20, 1),
    nn.Sigmoid()
)
                

2. Automatic Forward Pass

When you call:

output = model(x)
                

PyTorch automatically sends the data through all layers in order. You don't need to write a custom forward() function.

3. Great for Prototyping or Simple Models

Ideal for:

• Small feedforward neural networks
• Simple autoencoders
• CNNs without branching
• Rapid prototyping

When Should You Not Use It?

1. Limited Flexibility

You cannot easily:

• Reuse outputs from earlier layers (skip connections)
• Branch or merge network paths
• Apply conditional logic
• Use custom operations between layers (reshape, concat, math)

Example — not possible with plain Sequential:

# Not possible in nn.Sequential:
x1 = self.layer1(x)
x2 = self.layer2(x1)
out = x1 + x2  # skip connection
                

2. Hard to Debug or Customize

If you need intermediate outputs for debugging or visualization, nn.Sequential gives less control.

3. Not Suitable for Complex Architectures

These require custom forward passes:

• ResNet
• Transformers
• U-Net
• RNNs with loops or attention

Reference

• ChatGPT
• Perplexity
• PyTorch Documentation