Market Data Server

The Botmain MDS is a market data collection server allowing you to mass-collect, and use locally, market data from various centralized and decentralized exchanges.

We believe in empowering our users to have full control of their trading environments, and therefore do not charge for streaming data like many other platforms. Cloud-based processing creates inefficiencies and drives up latency and costs. Instead, our solution will provision an instance of the chosen underlying database software on your local system and allow you to mass-collect your own data to power your testing and strategies. Thanks for advances in GPU-based processing, future features of Botmain will leverage massively-parallel, localized backtesting and optimization, without the spike in costs of would-be cloud-based solutions.

The Botmain MDS integrates with the recently open-sourced TimeBase, an ultra-high speed data warehousing and real-time event processing/streaming server, battle-tested in commercial banking environments for over a decade and having absolutely unmatched performance.

Since TimeBase is a Linux or Windows only software, we also support the fully cross-platform Timescale! Though potentially less performant than TimeBase in certain cases, Timescale is perfectly sufficient for most production scenarios. For High-Frequency Trading (HFT), we still recommend TimeBase.

Datastreams

Datastreams in Botmain are configurable containers that can hold any type of data, from candles, ticks, & BBO to blockchain mempool data and even fundamental news data. Once price data is flowing into your streams, it is indexed on your computer for fast access and can be explored visually by filtering symbols to "MDS Markets" when selecting markets.

Datastreams combine historical and live streaming data into one continous stream that can be subscribed to by any numbers of your trading strategies. Strategies can "warm-up" on historical data until enough data is processed to begin making trading decisions, then seamlessly switch to streaming live data.

Collecting Data

1. Install an underlying database system:
   • Select to install TimeBase or Timescale depending on your needs and operating system. On Linux & Windows we recommend TimeBase.

2. Select the stream type:
   • Choose from the available stream type based on the type of data you want to collect

3. Configure the stream properties:
   • Choose which properties of data you want to collect into this stream. To make this easier, we offer predefined groups of properties based on the stream type.

4. Select symbols for data collection:
   • Selecting your newly created stream will display the overview of data in the stream, which should currently show as empty for all symbols. Using the checkboxes, find and select the instruments you want to collect data for.

5. Set stream specific settings and start the MDS:
   • Click "Collect Data" to open the data collection settings. Set the start date for data collection for the selected symbols. If data already exists for any of the selected symbols, the MDS will intelligently download only the parts it needs starting from the given start date.

6. Start MDS to begin collecting data into your stream:
   • Click "Start MDS" to begin collecting data