Backend worker that will periodically poll CCP Datasources for market prices and publish internally.
Still very much a work in progress but the idea is fairly complex.
Some constraints I'm putting on it:
- It must be threaded
- Must be able to download multiple pages concurrently
- It must be poolable
- Meaning it must have a way for something to limit how much it can suck on the internets
- Must be asynchronous
- The transactions it's downloading are not sticking around in memory for very long
- The state of the regions download must be known and publishable
- Meaning it can send a message when it starts and has completely finished