江苏龙网

  1. 主页 > 生活百科 > >

基于OGG 实现Oracle到Kafka增量数据实时同步( 四 )

Oracle

5.目标端开启实时数据同步
./ggsci GGSCI (172-16-101-242) 7> start replicat repkfk Sending START request to MANAGER ... REPLICAT REPKFK startingGGSCI (172-16-101-242) 8> info all Program     Status      Group       Lag at Chkpt  Time Since ChkptMANAGER     RUNNING REPLICAT    RUNNING     REPKFK      00:00:00      00:00:006.测试增量数据同步
Oracle插入增量数据
SQL> insert into baiyang.ora_to_kfk  select OWNER, OBJECT_NAME, SUBOBJECT_NAME, OBJECT_ID, DATA_OBJECT_ID, OBJECT_TYPE from all_objects where object_id >500 and  object_id < 1000; SQL> commit; SQL> select count(*) from baiyang.ora_to_kfk;COUNT(*) ----------     905查看Kafka消息队列消费数据
{"table":"BAIYANG.ORA_TO_KFK","op_type":"I","op_ts":"2019-11-11 21:04:11.158786","current_ts":"2019-11-11T21:10:54.042000","pos":"00000000000000075298","after":{"OWNER":"SYS","OBJECT_NAME":"APPLY$_READER_STATS","SUBOBJECT_NAME":null,"OBJECT_ID":998,"DATA_OBJECT_ID":998,"OBJECT_TYPE":"TABLE"}} {"table":"BAIYANG.ORA_TO_KFK","op_type":"I","op_ts":"2019-11-11 21:04:11.158786","current_ts":"2019-11-11T21:10:54.042001","pos":"00000000000000075459","after":{"OWNER":"SYS","OBJECT_NAME":"APPLY$_READER_STATS_I","SUBOBJECT_NAME":null,"OBJECT_ID":999,"DATA_OBJECT_ID":999,"OBJECT_TYPE":"INDEX"}}源端Oracle删除数据
SQL> delete from baiyang.ora_to_kfk ; 906 rows deleted. SQL> commit;查看kafka消息队列消费数据
{"table":"BAIYANG.ORA_TO_KFK","op_type":"D","op_ts":"2019-11-11 21:13:11.166184","current_ts":"2019-11-11T21:13:17.449007","pos":"00000000000000216645","before":{"OWNER":"x1","OBJECT_NAME":"SSSSS","SUBOBJECT_NAME":"z1","OBJECT_ID":111000,"DATA_OBJECT_ID":2000,"OBJECT_TYPE":"x1"}}源端插入数据
SQL> insert into  baiyang.ora_to_kfk values('汉字', 'y1', 'z1', 111000,2000,'x1'); 1 row created. SQL> commit;查看kafka消息队列消费数据
{"table":"BAIYANG.ORA_TO_KFK","op_type":"I","op_ts":"2019-11-11 21:14:21.167454","current_ts":"2019-11-11T21:14:26.497000","pos":"00000000000000216794","after":{"OWNER":"汉字","OBJECT_NAME":"y1","SUBOBJECT_NAME":"z1","OBJECT_ID":111000,"DATA_OBJECT_ID":2000,"OBJECT_TYPE":"x1"}}总结使用OGG可以方便地将Oracle的数据变更情况实时同步到Kafka消息队列 。下游业务系统通过订阅kafka的消息队列，能方便地实现各类实时数据的应用 。




推荐阅读


上一篇：Python爬虫遇到验证码的几种处理方式，文章末尾有源码

下一篇：聊聊HBase分布式数据库设计那些事