Bifrost Document

Version 1.6.x

Bifrost ---- A heterogeneous middleware which can synchronize MySQL binlog data To Redis,MongoDB,ClickHouse...

Recovery

<– 文件队列在数据写入的时候 ,是写到内核就认为成功了,由内核决定什么时候磁盘到磁盘,所以Bifrost保证不了 文件队列里的数据绝对正常

由内核决定什么时候磁盘,这样应用层就保证不了数据绝对正确

所以 Bifrost 启动的时候,会进行验证一次数据是否正确

Bifrost 在每次将数据写到文件队列的时候 ,都会进行标记一次进入到当前这个同步的最后位点信息,这个位点信息 是每5秒写到leveldb中去, 请参考 位点控制管理

每次启动,都将从文件队列中读取最后一条数据 和 当前这个同步配置最后位点 进行对比,如果一致的情况下,则认为当前这个文件队列的数据是可靠的,那这份数据,就继续使用

否则将放弃文件队列中的数据,通过 位点控制管理 从 MySQL Binlog 中重新解析 –>

When a file queue writes data to the kernel, it is considered successful when the data is written to the kernel. The kernel decides when the disk to disk is sent. Therefore, Bifrost cannot guarantee that the data in the file queue is absolutely normal.

When Bifrost starts, it will verify whether the data is correct.

Bifrost writes the data to the file queue, it will save the binlog position to leveldb.

The file queue is started, the last data will be read from the file queue and compared with the last binlog position of the current synchronization configuration. If the data is consistent, the data of the current file queue is considered reliable, and the data will continue to be used.

Otherwise, the data in the file queue will be discarded and re parsed from MySQL binlog through last success binlog position.

Last updated on 1 Nov 2020
Published on 1 Nov 2020
Edit on GitHub