Avoiding pitfalls when using cURL in CI/CD pipelines

Continuous Integration/Continuous Delivery (or Deployment, depending on your point of view) pipelines are at the core of many successful software projects. When designing your pipelines you sooner or later end up using REST calls to perform certain tasks. cURL is a popular command line tool to invoke REST APIs, and is commonly used in pipelines. Before you start using cURL in that capacity I’d like to draw your attention to a potential pitfall you can run into.

CI/CD Pipelines

A CI/CD pipeline typically consists of a series of tasks executed after a (git) commit is pushed to the remote registry. The idea is to ensure compliance with coding standards, formatting, and code quality, amongst a great wealth of other things. A pipeline is typically sub-divided into stages such as “build”, “lint”, “deploy” or anything else you can think of. Each stage consists of one or more tasks.

Whether or not the pipeline progresses to the next stage depends on the success or failure of tasks. Return codes are usually used to determine success or failure: a return code of 0 implies success, everything else usually terminates the pipeline’s execution.

Experimenting with cURL Exit Codes

In order to use cURL effectively in a CI pipeline it’s important to understand its error codes. Consider the following simulated API using node and express.js:

import express from 'express'
const app = express()
const port = 8080
const host = ''

// allow for a successful test
app.get('/', (req, res) => {
  res.set('Content-Type', 'text/plain')
  res.send('test successful')

// invoke this URL to provoke a HTTP 400 (bad request) error
// see https://expressjs.com/en/4x/api.html#res.set for details
app.get('/failure', (req, res) => {
  res.set('Content-Type', 'text/plain')
  res.status(400).send('Bad Request')

app.listen(port, host, () => {
  console.log(`Simulated API server available on ${host}:${port}!`)

I created a small container image with the above code using the node:lts image (that’s node 18.14.2 and express 4.18.2, the most current versions at the time of writing, Feb 25th) and ran it.

“But what about security?” I hear you ask. You will undoubtedly have noted that this isn’t production code, it lacks authentication and other security features, logging, and basically everything apart from returning a bit of text and a HTTP status code. I’m also going to use HTTP calls for the API – enabling HTTPS would have been overkill for my example. In the real world you wouldn’t run APIs without TLS protection, would you? Since this post is about HTTP status codes and cURL in CI pipelines none of the extra bells and whistles are necessary, and crucially they’d probably distract from the actual problem. If you’re coding your APIs you should always adhere to industry best practices!

Starting the container

I stated the container as follows:

podman run --rm -it --name some-api --publish 8080:8080 api:0.5

The CMD directive in the project’s Dockerfile starts node and passes the api.mjs file to it. The API is now ready for business:

Simulated API server available on!

Scenario 1: normal, successful completion

Let’s start with the successful invocation of the simulated API:

$ curl http://localhost:8080
test successful
$ echo $?

OK, nothing to see here, moving on… This is what was expected and shown for reference ;)

Scenario 2: Bad Request

I’m pointing curl to http://localhost:8080/failure next:

$ curl http://localhost:8080/failure
Bad Request
$ echo $?

Hmm, so that’s odd, curl‘s return code is 0 (= success) despite the error? Let’s dig a little deeper by using the verbose option and returning the headers

$ curl -iv http://localhost:8080/failure
*   Trying ::1:8080...
* Connected to localhost (::1) port 8080 (#0)
> GET /failure HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.74.0
> Accept: */*
* Mark bundle as not supporting multiuse
< HTTP/1.1 400 Bad Request
HTTP/1.1 400 Bad Request
< X-Powered-By: Express
X-Powered-By: Express
< Content-Type: text/plain; charset=utf-8
Content-Type: text/plain; charset=utf-8
< Content-Length: 11
Content-Length: 11
< ETag: W/"b-EFiDB1U+dmqzx9Mo2UjcZ1SJPO8"
ETag: W/"b-EFiDB1U+dmqzx9Mo2UjcZ1SJPO8"
< Date: Sat, 25 Feb 2023 11:34:16 GMT
Date: Sat, 25 Feb 2023 11:34:16 GMT
< Connection: keep-alive
Connection: keep-alive
< Keep-Alive: timeout=5
Keep-Alive: timeout=5

* Connection #0 to host localhost left intact
Bad Request

So it’s pretty clear that the HTTP status code is 400 (Bad Request). But that’s not reflected in the return code. Let’s fix this!

Instructing cURL to fail

A look at the cURL manual page reveals this interesting option:

       -f, --fail
              (HTTP) Fail silently (no output at all) on server  errors.  This
              is  mostly done to enable scripts etc to better deal with failed
              attempts. In normal cases when an HTTP server fails to deliver a
              document,  it  returns  an HTML document stating so (which often
              also describes why and more). This flag will prevent  curl  from
              outputting that and return error 22.

              This  method is not fail-safe and there are occasions where non-
              successful response codes will slip through, especially when au‐
              thentication is involved (response codes 401 and 407).

Which looks like exactly what I need. Let’s try this option:

$ curl --fail http://localhost:8080/failure
curl: (22) The requested URL returned error: 400 Bad Request
$ echo $?

Well that’s better! There’s a non-zero return code now.


The --fail option in curl (or --fail-with-body if your version of curl is 7.76 or later) allows DevOps engineers to architect their pipelines with greater resilience. Rather than manually parsing the cURL output checking for errors you can now rely on the REST API call’s return code to either proceed with the pipeline or stop execution. Please note that the –fail option isn’t fail-safe, as per the above comment in the man-page. Neither does it protect you from an API returning a HTTP-200 code if in fact an error occurred. But it’s definitely something I’ll use from now on by default.