Understanding Parallel Jobs in Azure DevOps: A Step-by-Step Guide

Hey there, cloud enthusiasts! Today, let's chat about something that might save your DevOps team a ton of time (and maybe a few headaches) - parallel jobs in Azure DevOps. If you've ever found yourself waiting impatiently for builds and releases to finish, this one's for you!

What Exactly Are Parallel Jobs, Anyway?

Imagine you're in a busy kitchen. If you have just one chef, they can only work on one dish at a time. But add a few more chefs, and suddenly multiple dishes can be prepared simultaneously. That's essentially what parallel jobs do in Azure DevOps!

In Azure-speak, a parallel job represents the capacity to run one build or release job at a time. With more parallel jobs, your system can juggle multiple pipelines simultaneously, rather than making each one wait its turn.

There are two main flavors of parallel jobs in Azure DevOps:

1. Microsoft-hosted Parallel Jobs

These run on machines that Microsoft maintains in the cloud. Think of them as professional kitchens that are ready to go - you just show up and cook. You don't worry about maintaining the stoves, replacing knives, or cleaning up afterward.

2. Self-hosted Parallel Jobs

These run on your own agent machines that you set up and maintain. It's like cooking in your own kitchen - you have complete control over the equipment and environment, but you're responsible for keeping everything running smoothly.

Why Should You Care About Parallel Jobs?

Let's be real - nobody likes waiting. With more parallel jobs, you can:

• Run multiple builds and deployments simultaneously

• Drastically reduce pipeline wait times

• Keep your developers productive (and happy!)

• Accelerate your delivery cycle

  
  

The Free Tier: What You Get Without Paying

Microsoft is pretty generous with the free stuff:

• Azure DevOps Services: You get one free Microsoft-hosted parallel job that can run up to 60 minutes per pipeline job, and one free self-hosted parallel job.

• Azure DevOps Server: You get one free self-hosted parallel job.

  
  

But let's face it, for any serious development team, one job isn't going to cut it. That's when you need to think about registering for more.

How to Register for More Parallel Jobs

For Microsoft-hosted Parallel Jobs:

• Log into Azure DevOps: Navigate to your organization (e.g., https://dev.azure.com/yourorganization)

  
  

• Access Organization Settings: Click on the Organization Settings gear icon in the bottom left

  
  

• Navigate to Parallel Jobs: Go to Pipelines → Parallel jobs

  
  

• Purchase Additional Capacity: Click on the "Change" button next to Microsoft-hosted jobs, and select how many additional parallel jobs you need

  
  

• Confirm Your Purchase: Review your selection and confirm

  
  

The pricing (as of this writing) is $40 per parallel job per month, and each provides an additional 1,800 minutes of build/release time per month.

For Self-hosted Parallel Jobs:

• Follow Steps 1-3 Above: Get to the Parallel Jobs page

  
  

• Purchase Self-hosted Jobs: Click "Change" next to Self-hosted jobs, and select your desired quantity

  
  

• Confirm Your Purchase: Review and complete the transaction

Self-hosted jobs currently cost $15 per parallel job per month, which is cheaper than Microsoft-hosted but remember - you're providing and maintaining the infrastructure.

Setting Up Self-hosted Agents:

If you opt for self-hosted jobs, you'll need to set up your own agents:

• Create a New Agent Pool: In Organization Settings → Agent Pools → Add Pool

  
  

• Download the Agent: Select the appropriate agent for your OS (Windows, Linux, or macOS)

  
  

• Configure the Agent:

# Extract the agent to a folder

mkdir myagent && cd myagent

tar zxvf ~/Downloads/vsts-agent-linux-x64-2.xxx.x.tar.gz

# Configure the agent

./config.sh

• Connect to Azure DevOps: You'll be prompted for:

• Your Azure DevOps organization URL

1.A personal access token (PAT)

2.The agent pool name

3.A unique agent name

• Start the Agent:

# Run as a service (recommended)

sudo ./svc.sh install

sudo ./svc.sh start

# Or run interactively

./run.sh

Pro Tips for Managing Parallel Jobs

• Monitor Usage: Keep an eye on your parallel job consumption to avoid unnecessary expenses

• Schedule Wisely: Try to schedule non-urgent pipelines during off-peak hours

• Use Deployment Groups: For deployment jobs, consider using deployment groups which don't consume parallel jobs

• Clean Agent Regularly: For self-hosted agents, regular maintenance keeps things running smoothly

  
  

When to Scale Up Your Parallel Jobs

You probably need more parallel jobs if:

• Builds are frequently queued and waiting

• Your team consistently complains about pipeline delays

• You're expanding your team or number of projects

• You're adopting a microservices architecture with many small repos and pipelines

• 
  

Wrapping Up

Parallel jobs are the secret sauce that can turn a sluggish DevOps pipeline into a well-oiled machine. Whether you choose Microsoft-hosted for convenience or self-hosted for customization and cost savings, adding more parallel capacity can significantly boost your team's productivity.

Remember, the goal is to reduce wait times and keep your development process flowing smoothly. Start with what you need, monitor usage, and scale as your team grows!