I have revived an old project to make SWISH more robust using horizontal scaling. That is still work in progress. Steps:
- Use Redis to allow SWISH instances to realize a shared view on the world and send each other messages. That was done some years ago, but taking the project to the next level stalled.
- Create a High availability with Redis Sentinel network and extend SWI-Prolog’s Redis interface to deal to TLS and Redis Sentinels.
- Migrate SWISH to use this infrastructure and run SWISH in a Docker container for easier deployment. That is now done with the primary public SWISH instance.
- Launched the instance swish2.swi-prolog.org. You can use it already, but be aware that it is restarted much more frequently than swish.swi-prolog.org during development. Note that the instances share saved documents, profiles, messages, etc.
That is where we are right now. This implies swish2.swi-prolog.org is available as backup and a possible fatal failure of one of the servers no longer results in data loss.
- Allow a SWISH client to switch to a different backend. There is already a menu item File/Backends … You can already manually select a different backend. Unfortunately only a small part of the functionality actually supports this now. Eventually, the system should automatically choose the best backend and switch the current backend fails or maybe even if it becomes too crowded.
- Have a failover mechanism that ensures we can always get the initial page.
- Reduce load on Redis. Some caching or smarter data structures are required to reduce the number of requests.
Main system is running again. There are some stability issues. Initially the health checking malfunctioned. That is hopefully fixed.