Guide to Using cURL with Proxy Servers

What is cURL?

cURL, short for "Client URL", is a tool for transferring and receiving data over the internet using URLs. As a software project, it consists of a library (libcurl) and a command-line tool (curl). Here we focus on curl, the command-line tool for data transfer using URL syntax.

The project supports a wide range of protocols, including HTTP, HTTPS, FTP, FTPS, SFTP, POP3, POP3S, IMAP, IMAPS, LDAP, LDAPS, SCP, and others. This makes curl the most popular and widely used command-line HTTP client in the world.

cURL is highly versatile and serves as a general solution for performing various tasks such as making HTTP requests, downloading files, uploading data, and interacting with APIs.

Installing curl

Now let's learn how to install curl on your machine.

macOS

On macOS, there is no need to install curl. The tool is already included in the operating system, and you can use it in the Terminal application.

Windows

Starting with Windows 10, Windows comes with a copy of curl. However, the curl command is an alias for the PowerShell command Invoke-WebRequest. This means that curl commands in the Windows terminal will invoke Invoke-Request behind the scenes. To avoid this and actually use curl from cURL, replace "curl" with "curl.exe". This way, PowerShell will run curl instead of Invoke-Request.

For example, to check the current version of curl installed on a Windows machine, use the following command in the terminal:

curl.exe --version

The output should be something like:

curl 8.0.1 (Windows) libcurl/8.0.1 Schannel WinIDN

Release date: 2023-03-20

Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp

Features: AsynchDNS HSTS HTTPS-proxy IDN IPv6 Kerberos Largefile NTLM SPNEGO SSL SSPI threadsafe Unicode UnixSockets

If you are a Windows user, replace all instances of "curl" in the article commands with "curl.exe". Otherwise, install Windows Subsystem for Linux (WSL) and follow the instructions below.

Linux

On Linux, the installation procedure for curl varies depending on your distribution. Popular Linux distributions like Ubuntu and Fedora come with it by default. Thus, you can use curl directly in the terminal.

In other distributions, curl may not be included. In that case, you can add it using the distribution's package manager. On Debian-based systems, you can install curl with the following command:

sudo apt-get install curl

What you need to use a proxy with curl

A proxy acts as an intermediary between the client and the destination server. It intercepts the client's requests, forwards them to the server, receives the server's response, and sends it back to the client. This intermediary approach enhances anonymity and helps bypass network restrictions. This is because the destination server will see requests as coming from the IP and location of the chosen proxy server, not from you.

To start using curl with proxy services, you first need access to a proxy server. Specifically, here is the syntax of a proxy server URL:

[<PROTOCOL>://][<USERNAME>:<PASSWORD>]@<HOST>[:<PORT>].

It consists of:

• <PROTOCOL>: The protocol to use for connecting to the proxy server. If not specified, curl defaults to http://.
• <HOST>: The required IP address or URL hostname of the proxy server.
• <PORT>: The port number the proxy server is listening on. If not specified, curl defaults to 1080.
• <USERNAME>: An optional username required for authentication.
• <PASSWORD>: An optional password required for authentication.

As for proxy protocols, HTTP and HTTPS are the most popular, followed by SOCKS.

It's time to get a working HTTP proxy!

You can obtain