2014년 6월 21일 토요일

Memcached Replication (1)

Purpose

A lot of applications have data which have to be stored persistently. And theses data is stored on RDBMS or NoSQL solutions.
Sometimes these applications need a lot of data search operations, but current RDBMS or NoSQL solutions can't serve this requirement.
RDMBS or NoSQL solutions do really complicated internal processing for applications' request, and they have to store data to disk slowest component of computer.
So we use memory cache solution like Redis or Memcached. But they also have weak points especially Memcached doesn't have repilcation features.
Redis, this cache solution also have some weak points, this is not why I am focusing Memcached in this article.
We use a lot of Memcached on our service already, and I have to make Memcached can replicate data to remote Memcached(like slave of mysql) without migrating to Redis.

Some people doesn't feel any needs for Memcached replication. 
But replication feature is necessary for multi-idc synchronization. And multi-idc synchronization is need for disaster recovery or IDC location aware services.
Redis has replication features, but Memcached server has not. Redis can serve complex data types but Memcached not. On the other hand, Memcached has it's own advantages.
(I don't want to metion about "Why Memcached is better than Redis, and Why Redis better than Memcached", What I want to say in this article is there's still a lot of people use Memcached server).
Some company made mysql server's binary log parser and relay it to Memcached server of multi IDC. Because usually cached data items are originated from database server.
And this way, solve a lot of complexity of application.

Also our company(Kakao corp) has same needs for DR or location aware services.
But above binary log parsing method need a few change of sql statement of applications. It's not fully transparent from application.


MySQL 5.6 Memcached plugin

MySQL 5.6 is released during I am looking for memcached replication method, First time I saw the MySQL Memcached plugin it seems that it doesn't have no usability.
But after a few days, Suddenly I thought we use MySQL Server's Memcached plugin as standalone memcached server and easily have memcached replication. Because Memcached data change will be written to mysql binary log through MySQL(InnoDB API). MySQL Memcached plugin has below three cache policy.


  • Innodb-only
  • Caching
  • Cache-only


First of all, Memcached plugin act as whole memory operation. So we could not use Inoodb-only and Caching policy because memcached data will be stored in innodb finally on this two policy. We have to use Cache-only policy. But unfortunately Memcached data is never written to binary log because memcached data is not stored in InnoDB on Cache-only policy.
And this is only the story for binary log writting. But on slave side SQL thread will relay binary log contents to InnoDB only, Not memcached plugin.

If I changed MySQL as writting data change to binary log on master and apply change to memcached plugin on slave, Memcached replication can be possible.
This is not so easy task, but it would be best way to implement it. We don't have to implement whole replication features on Memcached and MySQL replication features is really stable.

댓글 없음:

댓글 쓰기