# Docker API example **URL:** \ **Category:** Nice to know\ **Created:** [November 11, 2020, 7:53pm UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252 "2020-11-11T19:53:27Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [November 11, 2020, 7:53pm UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252/1 "2020-11-11T19:53:27Z")
Most service APIs that use [JSON](https://en.wikipedia.org/wiki/JSON) use [TCP/IP](https://en.wikipedia.org/wiki/Internet_protocol_suite) over a [Network Socket](https://en.wikipedia.org/wiki/Network_socket). However the [Docker Engine API](https://docs.docker.com/engine/api/v1.40/#) can also use a [Unix Domain Socket](https://en.wikipedia.org/wiki/Unix_domain_socket) (UDS) but then uses [HTTP](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol) with the UDS. Since using UDS with HTTP, as demonstrated in the Docker API [examples](https://docs.docker.com/engine/api/sdk/examples/), does not have an off the shelf SWI-Prolog example this took a bit of effort to figure out. ```prolog docker_engine_api_uds_http_example :- unix_domain_socket(Socket), File = "/var/run/docker.sock", tcp_connect(Socket, File), tcp_open_socket(Socket, Stream), Request = "GET /images/json HTTP/1.0\r\n\r\n", format(Stream, '~s', [Request]), flush_output(Stream), read_string(Stream,"","",_,Response), close(Stream), % format('~w~n',[Response]), assertion(sub_string(Response,_,_,_,"HTTP/1.0 200 OK")). ``` 🙂 ---
**Author:** ![panasenco](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/panasenco/32/3172_2.png) [@panasenco](https://swi-prolog.discourse.group/u/panasenco)\ **Post date:** [November 11, 2020, 9:56pm UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252/2 "2020-11-11T21:56:36Z")
This is very cool, Eric! I currently use a small service that’s based on this functionality (exposing /var/run/docker.sock) called [swarm-cronjob](https://crazymax.dev/swarm-cronjob/). It’s a service that just starts Docker containers at a given time. Very useful as some of our services are not allowed to run during certain times of day or night. It’s great to know that a similar functionality can be implemented in Prolog for custom use cases! Aram ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [November 12, 2020, 8:14am UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252/3 "2020-11-12T08:14:21Z")
I guess we should add support to http\_open/3 and the HTTP server library for Unix domain sockets. Shouldn’t be too hard as the establishing the socket is already hookable in both libraries to make HTTPS work. It is mostly a matter of specifying that you want a Unix domain socket. For the server there is nothing standard and thus we could use e.g. unix(Path) in addition to either Port or IFace:Port. For clients I’ve seen `http+unix://Path/Request`, where Path is a URL-Encoded file name (notably using `%2F` for /). See [https://github.com/msabramo/requests-unixsocket](https://github.com/msabramo/requests-unixsocket). I consider this a bit dubious as + is not a valid character in a URI scheme AFAIK and thus the thing is not a valid URI. Curl seems to split the options, so we have ([https://github.com/aelsabbahy/goss/issues/303](https://github.com/aelsabbahy/goss/issues/303)) ``` curl -gG --unix-socket /var/run/docker.sock http://localhost/containers/json ``` (edited; original says `http:/loclahost`, but that seems wrong to me). That also seems a bit strange to me as that will make a redirect to server on `localhost` ambiguous. Any other proposals? ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [November 12, 2020, 10:09am UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252/4 "2020-11-12T10:09:20Z")
> [@jan](#): > > I guess we should add support to [http\_open/3](https://www.swi-prolog.org/pldoc/doc_for?object=http_open/3) and the HTTP server library for Unix domain sockets. Yes! I was hoping this post would trigger you to shine your expertise on this. > [@jan](#): > > to make HTTPS work. Why would someone want to use HTTPS with a Unix Domain Socket? Since the communication is all within an OS kernel why would there be a need to encrypt and decrypt it? > [@jan](#): > > For the server there is nothing standard and thus we could use e.g. unix(Path) in addition to either Port or IFace:Port. As I only started to understand the details of UDS in trying to get this code to work, I am lost in trying to understand that sentence. > [@jan](#): > > Any other proposals? I would like to be able to intelligently add to this, but being so new to using UDS I think anything I were to suggest at present would be more out of need for a specific problem and not useful in the general sense. But as I learn more I will keep my eyes open. I would hope those who liked the original [post](https://swi-prolog.discourse.group/t/docker-api-example/3252) would add to this. As I have communicated with each of them in detail in the past on such things I know they have much more valuable and real world experience than me with when using UDS. * * * Also I am changing the category of this from [Useful Code](https://swi-prolog.discourse.group/c/useful-code/16) to [Nice to know](https://swi-prolog.discourse.group/c/nice-to-know/17) as Useful code should be more refined and does not allow discussions and this code clearly needs to be discussed. ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [November 12, 2020, 10:30am UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252/5 "2020-11-12T10:30:01Z")
> [@EricGT](#): > > Why would someone want to use HTTPS with a Unix Domain Socket? You don’t. You do like to reuse the low-level facility to hook how the socket is established. > [@EricGT](#): > > I am lost in trying to understand that sentence. The predicate http\_server/1 from library(http/htp\_server) must somehow know what server socket to create. As is, this is port(Port) or port(IFace:Port) to either specify a simple TCP port or a port bound to a specific interface (typically `localhost`). port(unix(Path)) could be used, but reconsidering that seems a little weird. Probably unix(Path) instead of port(…) is more adequate. ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [November 12, 2020, 10:45am UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252/6 "2020-11-12T10:45:36Z")
> [@jan](#): > > unix(Path) That makes more sense. However I think someone not understanding file\_search\_path/2 might think that will open a Unix file, when the meaning here is to open a Unix Domain Socket. Does `unix_domain_socket(Path)` make more sense? Also if someone new to SWI-Prolog were to see `unix(Path)` they might then look for a similar `windows(Path)` and not understand why one exist and not the other. As odd as this may sound, in almost 40 years of programming I never really understood as much about UDS until just this last week. I missed the details of the signifigant difference over all of these years because I mainly program in Windows, I am sure others would not see the difference between a Network Socket and a Unix Domain Socket until they have to work with them. * * * **EDIT** In doing more research related to using the Docker API found [this](https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-socket-option) which talks about how to start up the Docker daemon with options for using different sockets. There are much more than just this one ```prolog # listen using the default unix socket, and on 2 specific IP addresses on this host. $ sudo dockerd -H unix:///var/run/docker.sock -H tcp://192.168.59.106 -H tcp://10.10.10.2 ``` * * * **EDIT** For anyone trying to figure out the specifics of the Docker HTTP GET (request) that is needed for each command, spying on the data as it passes through the UDS is beneficial. See: [Can I monitor a local unix domain socket like tcpdump?](https://superuser.com/questions/484671/can-i-monitor-a-local-unix-domain-socket-like-tcpdump) The essence of the [solution](https://superuser.com/a/576404/125532) is ```bash groot@Galaxy:~$ sudo mv /var/run/docker.sock /var/run/docker.sock.original groot@Galaxy:~$ sudo socat -t100 -x -v UNIX-LISTEN:/var/run/docker.sock,mode=777,reuseaddr,fork UNIX-CONNECT:/var/run/docker.sock.original ``` To use this, start one terminal up and use the above commands, then start up a second terminal to send the command using [CURL](https://curl.se/) as demonstrated in the [Docker API examples](https://docs.docker.com/engine/api/sdk/examples/). The result will be a hexdump in the first terminal as the commands are executed. ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [November 12, 2020, 2:56pm UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252/7 "2020-11-12T14:56:20Z")
> [@jan](#): > > (edited; original says `http:/loclahost` , but that seems wrong to me). That also seems a bit strange to me as that will make a redirect to server on `localhost` ambiguous. In further reading [found](https://docs.docker.com/engine/api/sdk/examples/#run-a-container): > When using cURL to connect over a unix socket, **the hostname is not important**. The examples above use `localhost` , but any hostname would work. ---
**Author:** ![jamesnvc](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jamesnvc/32/14_2.png) [@jamesnvc](https://swi-prolog.discourse.group/u/jamesnvc)\ **Post date:** [November 12, 2020, 3:00pm UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252/8 "2020-11-12T15:00:57Z")
I was going to say, when I’ve used curl to a socket, I usually omit the host, so it would be like `curl --unix-socket foo.sock http:///foo/bar` ---
**Author:** ![jan](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/jan/32/4_2.png) [@jan](https://swi-prolog.discourse.group/u/jan)\ **Post date:** [November 12, 2020, 3:17pm UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252/9 "2020-11-12T15:17:40Z")
Thanks. That makes sense. So that would add some option to http\_open/3 to tell it to use a Unix domain socket and subsequently ignore the host (and port) from the URL. I think I like that better then the `http+unix://` approach. If we get a redirect I assume we should check the host is absent or the same as we used, in case which we keep using the Unix domain socket. If not we should drop the Unix domain socket option and do a normal HTTP request. Now the option name. I tend to agree that unix(Path) may lead to confusion. unix\_domain\_socket(Path) is a bit long IMO. `af_unix(Path)` could be an alternative as the protocol is named `AF_UNIX` and thus anyone searching docs or code will look for that. ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [November 12, 2020, 3:38pm UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252/10 "2020-11-12T15:38:00Z")
> [@jan](#): > > the protocol is named `AF_UNIX` and thus anyone searching docs or code will look for that. When I started trying to just understand `/var/run/docker.sock` and not knowing the difference between a Network Socket and and a Unix Domain Socket and just thinking they were essentially the same thing, it took me a while get that they were two different things. Then in searching found that `Unix Socket` is quite common along with other names like `Unix Domain Socket`, `AF_UNIX`, `AF_LOCAL`, `IPC socket`, `UDS`. But not everyone connected the different names together. Needless to say it kept me confused for quite some time. Also as you were confused buy the use of `localhost` in the Docker API example using cURL, it had me even more confused trying to find an example that used it. What finally helped to clear the confusion were the examples in this [gist](https://gist.github.com/nuxlli/7553996). So while I understand and would agree that `AF_UNIX` makes sense, can there be both `AF_UNIX` and `unix_domain_socket(Path)`? Or at least in the documentation note the differences between a Network Socket and and a Unix Domain Socket as I think many will not make the connection between `AF_UNIX` and `Unix Domain Socket` without further reading and understand the importance of using a Unix Domain Socket, namely that they are faster and more secure but also restricted to just one OS kernel. 🙂 * * * **EDIT** > [@jan](#): > > If we get a redirect I assume we should check the host is absent or the same as we used, in case which we keep using the Unix domain socket. If not we should drop the Unix domain socket option and do a normal HTTP request. Sorry I can’t be of more help with that. While I get the concept, my mind is stuck in only seeing how this works with the Docker API. From what I am finding, the only way to communicate with the Docker daemon is via the API, so in using the [Docker CLI](https://docs.docker.com/engine/reference/commandline/cli/) and spying on `/var/run/docker.sock` I am seeing the various messages they are using. While the Docker API examples would seem to suggest that it is a single command, which is true in the sense that a single command does the work, there are other ping and info messages executed before the actual command is sent. See: [Docker Overview](https://docs.docker.com/get-started/overview/) which explains this in more detail. * * * **EDIT** > **Example of capture** > > On terminal 1 > > ```plaintext > groot@Galaxy:~$ sudo mv /var/run/docker.sock /var/run/docker.sock.original > groot@Galaxy:~$ sudo socat -t100 -v UNIX-LISTEN:/var/run/docker.sock,mode=777,reuseaddr,fork UNIX- > > ``` > > Using Docker CLI on terminal 2 > > ```plaintext > groot@Galaxy:~$ docker pull alpine > Using default tag: latest > latest: Pulling from library/alpine > Digest: sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a > Status: Image is up to date for alpine:latest > docker.io/library/alpine:latest > > ``` > > What is captured (spy) and displayed on terminal 1 > > ```plaintext > CONNECT:/var/run/docker.sock.original > > 2020/11/12 14:01:49.239890 length=82 from=0 to=81 > HEAD /_ping HTTP/1.1\r > Host: docker\r > User-Agent: Docker-Client/19.03.13 (linux)\r > \r > < 2020/11/12 14:01:49.276652 length=282 from=0 to=281 > HTTP/1.1 200 OK\r > Api-Version: 1.40\r > Builder-Version: 2\r > Cache-Control: no-cache, no-store, must-revalidate\r > Content-Type: text/plain; charset=utf-8\r > Date: Thu, 12 Nov 2020 19:01:49 GMT\r > Docker-Experimental: false\r > Ostype: linux\r > Pragma: no-cache\r > Server: Docker/19.03.13 (linux)\r > \r > > 2020/11/12 14:01:49.644838 length=82 from=0 to=81 > HEAD /_ping HTTP/1.1\r > Host: docker\r > User-Agent: Docker-Client/19.03.13 (linux)\r > \r > < 2020/11/12 14:01:49.653505 length=282 from=0 to=281 > HTTP/1.1 200 OK\r > Api-Version: 1.40\r > Builder-Version: 2\r > Cache-Control: no-cache, no-store, must-revalidate\r > Content-Type: text/plain; charset=utf-8\r > Date: Thu, 12 Nov 2020 19:01:49 GMT\r > Docker-Experimental: false\r > Ostype: linux\r > Pragma: no-cache\r > Server: Docker/19.03.13 (linux)\r > \r > > 2020/11/12 14:01:49.937226 length=82 from=0 to=81 > HEAD /_ping HTTP/1.1\r > Host: docker\r > User-Agent: Docker-Client/19.03.13 (linux)\r > \r > < 2020/11/12 14:01:49.942193 length=282 from=0 to=281 > HTTP/1.1 200 OK\r > Api-Version: 1.40\r > Builder-Version: 2\r > Cache-Control: no-cache, no-store, must-revalidate\r > Content-Type: text/plain; charset=utf-8\r > Date: Thu, 12 Nov 2020 19:01:49 GMT\r > Docker-Experimental: false\r > Ostype: linux\r > Pragma: no-cache\r > Server: Docker/19.03.13 (linux)\r > \r > > 2020/11/12 14:01:49.944344 length=86 from=82 to=167 > GET /v1.40/info HTTP/1.1\r > Host: docker\r > User-Agent: Docker-Client/19.03.13 (linux)\r > \r > < 2020/11/12 14:01:50.140954 length=211 from=282 to=492 > HTTP/1.1 200 OK\r > Api-Version: 1.40\r > Content-Type: application/json\r > Date: Thu, 12 Nov 2020 19:01:50 GMT\r > Docker-Experimental: false\r > Ostype: linux\r > Server: Docker/19.03.13 (linux)\r > Transfer-Encoding: chunked\r > \r > < 2020/11/12 14:01:50.141676 length=2377 from=493 to=2869 > 942\r > {"ID":"ER7F:4CFQ:NF76:MKKG:J262:DJMW:H6KE:5K5S:WZZX:KRWP:GFI7:QBKH","Containers":3,"ContainersRunning":1,"ContainersPaused":0,"ContainersStopped":2,"Images":4,"Driver":"overlay2","DriverStatus":[["Backing Filesystem","extfs"],["Supports d_type","true"],["Native Overlay Diff","true"]],"SystemStatus":null,"Plugins":{"Volume":["local"],"Network":["bridge","host","ipvlan","macvlan","null","overlay"],"Authorization":null,"Log":["awslogs","fluentd","gcplogs","gelf","journald","json-file","local","logentries","splunk","syslog"]},"MemoryLimit":true,"SwapLimit":true,"KernelMemory":true,"KernelMemoryTCP":true,"CpuCfsPeriod":true,"CpuCfsQuota":true,"CPUShares":true,"CPUSet":true,"PidsLimit":true,"IPv4Forwarding":true,"BridgeNfIptables":false,"BridgeNfIp6tables":false,"Debug":false,"NFd":51,"OomKillDisable":true,"NGoroutines":64,"SystemTime":"2020-11-12T19:01:49.986345Z","LoggingDriver":"json-file","CgroupDriver":"cgroupfs","NEventsListener":6,"KernelVersion":"4.19.128-microsoft-standard","OperatingSystem":"Docker Desktop","OSType":"linux","Architecture":"x86_64","IndexServerAddress":"https://index.docker.io/v1/","RegistryConfig":{"AllowNondistributableArtifactsCIDRs":[],"AllowNondistributableArtifactsHostnames":[],"InsecureRegistryCIDRs":["127.0.0.0/8"],"IndexConfigs":{"docker.io":{"Name":"docker.io","Mirrors":[],"Secure":true,"Official":true}},"Mirrors":[]},"NCPU":4,"MemTotal":6576685056,"GenericResources":null,"DockerRootDir":"/var/lib/docker","HttpProxy":"","HttpsProxy":"","NoProxy":"","Name":"docker-desktop","Labels":[],"ExperimentalBuild":false,"ServerVersion":"19.03.13","ClusterStore":"","ClusterAdvertise":"","Runtimes":{"runc":{"path":"runc"}},"DefaultRuntime":"runc","Swarm":{"NodeID":"","NodeAddr":"","LocalNodeState":"inactive","ControlAvailable":false,"Error":"","RemoteManagers":null},"LiveRestoreEnabled":false,"Isolation":"","InitBinary":"docker-init","ContainerdCommit":{"ID":"8fba4e9a7d01810a393d5d25a3621dc101981175","Expected":"8fba4e9a7d01810a393d5d25a3621dc101981175"},"RuncCommit":{"ID":"dc9208a3303feef5b3839f4323d9beb36df0a9dd","Expected":"dc9208a3303feef5b3839f4323d9beb36df0a9dd"},"InitCommit":{"ID":"fec3683","Expected":"fec3683"},"SecurityOptions":["name=seccomp,profile=default"],"ProductLicense":"Community Engine","Warnings":["WARNING: bridge-nf-call-iptables is disabled","WARNING: bridge-nf-call-ip6tables is disabled"]} > \r > < 2020/11/12 14:01:50.150569 length=5 from=2870 to=2874 > 0\r > \r > > 2020/11/12 14:01:50.743315 length=252 from=168 to=419 > POST /v1.40/images/create?fromImage=alpine&tag=latest HTTP/1.1\r > Host: docker\r > User-Agent: Docker-Client/19.03.13 (linux)\r > Content-Length: 0\r > Content-Type: text/plain\r > X-Registry-Auth: eyJ1c2VybmFtZSI6ImNsb2NrcyIsInBhc3N3b3JkIjoiNXdbM2RHX2tKNyxZIn0=\r > \r > < 2020/11/12 14:01:51.383870 length=211 from=2875 to=3085 > HTTP/1.1 200 OK\r > Api-Version: 1.40\r > Content-Type: application/json\r > Date: Thu, 12 Nov 2020 19:01:51 GMT\r > Docker-Experimental: false\r > Ostype: linux\r > Server: Docker/19.03.13 (linux)\r > Transfer-Encoding: chunked\r > \r > < 2020/11/12 14:01:51.394980 length=62 from=3086 to=3147 > 38\r > {"status":"Pulling from library/alpine","id":"latest"}\r > \r > < 2020/11/12 14:01:51.726415 length=160 from=3148 to=3307 > 9a\r > {"status":"Digest: sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a"}\r > {"status":"Status: Image is up to date for alpine:latest"}\r > \r > < 2020/11/12 14:01:51.735361 length=5 from=3308 to=3312 > 0\r > \r > > ``` > > Notice the additional HTTP request, e.g. `_ping`, `info`, before the actual `POST` command that does the work. ---
**Author:** ![panasenco](https://yyz2.discourse-cdn.com/free1/user_avatar/swi-prolog.discourse.group/panasenco/32/3172_2.png) [@panasenco](https://swi-prolog.discourse.group/u/panasenco)\ **Post date:** [November 12, 2020, 5:40pm UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252/11 "2020-11-12T17:40:32Z")
Adding a vote for `unix_domain_socket` over `unix` or `af_unix`. IMO it’s OK for a rarely-used option to have a longer and more descriptive name. ---
**Author:** ![EricGT](https://avatars.discourse-cdn.com/v4/letter/e/f1d935/32.png) [@EricGT](https://swi-prolog.discourse.group/u/EricGT)\ **Post date:** [November 12, 2020, 7:35pm UTC](https://swi-prolog.discourse.group/t/docker-api-example/3252/12 "2020-11-12T19:35:30Z")
> [@jamesnvc](#): > > I was going to say, when I’ve used curl to a socket, I usually omit the host, so it would be like `curl --unix-socket foo.sock http:///foo/bar` Thanks. That is so much more understandable without `localhost`. I don’t know why the Docker API examples left `localhost` in the command even when they noted it is not used. I guess it was a hold over from earlier days when it may have been needed for cURL, but it clearly is not needed now. 🙂