Safe spool Safe spool is C++ library suitable for realtime queueing, persistent queues, spooling binary items, while giving guarantees for persistence. It has been optimized for putting things into the queue. It works by always writing the items to queue into a file, and the letteing a background thread forward the items. The queue file is always written and read sequentially which improves performance a lot. This sounds impossible but it is not. When the queue file has grown to a (configurable) size the library will start writing new items to an overflow file. When the background forwarding thread has forwarded all the items that are in the normal file but not in the overflow file if will switch to overflow file to the normal file and continue. Things to worry about: - If you forward-implementation pushes the items to the network then if the incoming rate exceeds the rate that you can push it through the network the queue file will grow and grow... - If you need an absolute guarantee that things have been written to the disc you need to fiddle with the O_DSYNC options to open() and create, or put the queue files on a "sync" mounted filesystem. For high loads don't do this unless you use a journaled filesystem (unless you like to make your disc sound like a chainsaw...) - The maximum throughput is limited by your disc bandwidth divided by 3. - Sometimes items are sent to your forwarding function more than once. Deal with it. Performance: My system has no problems with queueing 23MB/sec