How do I load data from an arbitrary file format into prolog?

You can write a simple parser for your file format using DCGs. For a nice tutorial, see:

http://www.pathwayslms.com/swipltuts/dcg/