{"componentChunkName":"component---src-templates-blog-post-js","path":"/blog/serve-a-json-resume-to-curl-with-caddy","result":{"data":{"allGhostPost":{"edges":[{"node":{"title":"Serve a JSON resume to cURL with Caddy","html":"

Open your terminal and run curl kamal.io. You should see a JSON "résumé." Browse to kamal.io in your browser however, and you'll get a regular site! What's going on?

\n
\n

How

\n

When a request is sent to kamal.io, the webserver, Caddy treats the curl client differently. It does so by checking the User-agent header and processing the request like so:

\n\n

This is what the relevant parts of my Caddy config look like:

\n
rewrite {\n    if {path} is "/"\n    if {>User-agent} starts_with curl\n    to /about.json\n}\n
\n

But

\n

This works, but there's one thing we need to take care of. Caddy configures a forced HTTP → HTTPS redirect by default. curl doesn't follow redirect headers unless the -L flag is passed. So, if you run curl kamal.io with only the config above, it won't work.

\n
~ ❯❯❯ curl kamal.io\n<a href="https://kamal.io/">Moved Permanently</a>.\n
\n

This works, however:

\n
~ ❯❯❯ curl -L kamal.io\n{\n    "name": "Kamal Nasser",\n    ...\n
\n

What Then

\n

To fix this issue, we need to configure Caddy to not redirect HTTP curl requests to / to HTTPS. To do so, you will need to add an HTTP server block, like so:

\n
http://kamal.io/ {\n    ...\n}\n
\n

This will let you override the default HTTP → HTTPS redirect and configure HTTP requests however you like. This matches all requests under / (e.g. /index.html), not only / itself.

\n

Similar to the rewrite directive above, we will need to add a redir directive that redirects only if the request isn't from a curl client and isn't exactly /.

\n
http://kamal.io/ {\n\tredir 301 {\n\t\tif {path} not "/"\n\t\tif {>User-agent} not_starts_with "curl"\n\t\tif_op or\n\t\t/  https://{host}{uri}\n\t}\n\n\t... webroot config ...\n}\n
\n

Now, it will be possible to run curl kamal.io as-is and get a proper response and get an HTTPS redirect for every other kind of request.

\n

But I'm using nginx. Can I still do this?

\n

Yes. I just switched to Caddy and had been using nginx before. This is what my config looked like:

\n
server {\n    server_name www.kamal.io kamal.io;\n    listen 80;\n    listen 443 ssl http2;\n\n    if ($http_user_agent ~* ^curl) {\n        set $redirect N;\n        rewrite ^/$ /about.json;\n        break;\n    }\n\n    # I had my website behind Cloudflare, so I needed to check the X-Forwarded-Proto header.\n    # If you're using plain Nginx, use if ($https = "") instead.\n    if ($http_x_forwarded_proto = "http") {\n        set $redirect "${redirect}Y";\n    }\n\n    if ($redirect = Y) {\n        return 301 https://kamal.io$uri;\n    }\n    \n    ... rest of config ...\n}\n
\n

Without HTTPS, you can use a simpler config:

\n
location / {\n    if ($http_user_agent ~* ^curl) {\n        rewrite ^/$ /about.json;\n        break;\n    }\n}\n
\n

Another option would be splitting your server blocks into separate HTTP and HTTPS server blocks (which I preferred when I used nginx) and then adapting the snippet above to them.

\n","published_at":"2018-07-19T20:20:08.000+03:00","slug":"serve-a-json-resume-to-curl-with-caddy","tags":[],"plaintext":"Open your terminal and run curl kamal.io. You should see a JSON \"résumé.\" Browse\nto kamal.io [https://kamal.io] in your browser however, and you'll get a\nregular site! What's going on?\n\nHow\nWhen a request is sent to kamal.io, the webserver, Caddy\n[https://caddyserver.com/] treats the curl client differently. It does so by\nchecking the User-agent header and processing the request like so:\n\n * if the User-agent begins with curl and the request path is /: * rewrite the\n request internally to /about.json\n \n \n * otherwise, serve the request as normal\n\nThis is what the relevant parts of my Caddy config look like:\n\nrewrite {\n if {path} is \"/\"\n if {>User-agent} starts_with curl\n to /about.json\n}\n\n\nBut\nThis works, but there's one thing we need to take care of. Caddy configures a\nforced HTTP → HTTPS redirect by default. curl doesn't follow redirect headers\nunless the -L flag is passed. So, if you run curl kamal.io with only the\nconfig above, it won't work.\n\n~ ❯❯❯ curl kamal.io\nMoved Permanently.\n\n\nThis works, however:\n\n~ ❯❯❯ curl -L kamal.io\n{\n \"name\": \"Kamal Nasser\",\n ...\n\n\nWhat Then\nTo fix this issue, we need to configure Caddy to not redirect HTTP curl\nrequests to / to HTTPS. To do so, you will need to add an HTTP server block,\nlike so:\n\nhttp://kamal.io/ {\n ...\n}\n\n\nThis will let you override the default HTTP → HTTPS redirect and configure HTTP\nrequests however you like. This matches all requests under / (e.g. /index.html\n), not only / itself.\n\nSimilar to the rewrite directive above, we will need to add a redir directive\nthat redirects only if the request isn't from a curl client and isn't exactly /\n.\n\nhttp://kamal.io/ {\n\tredir 301 {\n\t\tif {path} not \"/\"\n\t\tif {>User-agent} not_starts_with \"curl\"\n\t\tif_op or\n\t\t/ https://{host}{uri}\n\t}\n\n\t... webroot config ...\n}\n\n\nNow, it will be possible to run curl kamal.io as-is and get a proper response\nand get an HTTPS redirect for every other kind of request.\n\nBut I'm using nginx. Can I still do this?\nYes. I just switched to Caddy and had been using nginx before. This is what my\nconfig looked like:\n\nserver {\n server_name www.kamal.io kamal.io;\n listen 80;\n listen 443 ssl http2;\n\n if ($http_user_agent ~* ^curl) {\n set $redirect N;\n rewrite ^/$ /about.json;\n break;\n }\n\n # I had my website behind Cloudflare, so I needed to check the X-Forwarded-Proto header.\n # If you're using plain Nginx, use if ($https = \"\") instead.\n if ($http_x_forwarded_proto = \"http\") {\n set $redirect \"${redirect}Y\";\n }\n\n if ($redirect = Y) {\n return 301 https://kamal.io$uri;\n }\n \n ... rest of config ...\n}\n\n\nWithout HTTPS, you can use a simpler config:\n\nlocation / {\n if ($http_user_agent ~* ^curl) {\n rewrite ^/$ /about.json;\n break;\n }\n}\n\n\nAnother option would be splitting your server blocks into separate HTTP and\nHTTPS server blocks (which I preferred when I used nginx) and then adapting the\nsnippet above to them.","meta_description":null}}]}},"pageContext":{"slug":"serve-a-json-resume-to-curl-with-caddy","prev":"fresh-coffee-aeropress","next":"https-blog-digitalocean-com-deploying-a-fully-automated-git-based-static-website-in-under-5-minutes"}},"staticQueryHashes":["3649515864"]}