DB Update

September 19th, 2009 | by bryan |

serverThere was a bit of downtime tonight for some database maintenance. Unfortunately the size of the database has been growing¬† by leaps and bounds due to the hundreds of thousands of feeds being tracked and with daddy duty, I had not been staying on top of it. This afternoon I noticed an error that the db server filesystem was full. ReadPath’s DB server is a HP ML150 with 12G of ram and dual 150G disks.¬† The issue that I came across was that while there is nothing else really running on the server other than mysql, the ibdata file for the innodb tables was using 134G or 140 usable gigabytes. df -h was reporting 100% full on the disk. The primary issue with ibdata files is that they grow but don’t ever really shrink. Even if I deleted data out of the database it wouldn’t help my situation. The only fix is to export the data and then reinsert it which wouldn’t really work here.

So I cleared away some log files and other cruft which gave me 64M to make it till the evening with spidering turned off. Then tonight I put in an additional 1T sata drive that I had intended to use in the hadoop/hbase cluster. With a quick format of the disk, copy the ibdata file over to the new hard drive (which took 30 min for 134G), and then a link from the old ibdata location to the new one and we’re up and running.

Everything seems to check out, but let me know if you see any issues.




Post a Comment