When WSO2 APIM Analytics database is unusually growing

Photo by Tobias Fischer on Unsplash

I am writing this article assuming the readers are aware of what is WSO2, WSO2 API Manager and WSO2 APIM Analytics are.

Assume you have configured WSO2 APIM with APIM Analytics and you have been using the system for a considerable time. At this point, you may observer that analytics event database is growing in size as APIM is continuously publishing events to APIM Analytics server.

In APIM Analytics server, there are a few tables which store data and summarize them into another tables. I wouldn’t go to much details about them.

However, it is recommended to purge the data in these tables after a while. Because, the data in these tables are being summarized periodically. Hence the already summarized data would no longer be needed unless you have a special use of those raw data.

You can refer article on how to purge data in APIM Analytics for further details and it contains the recommended tables to purge and all the details related to purging. If you enable purging, then there would not be any data beyond a certain time period in the purging enabled tables.

You can check if data purging is happening as expected by going to Data Explorer in APIM Analytics server. There you can select the tables subjected to purging ( only limited numbers of tables are allowed to purge: refer the article for more info) and search by date range to check if tables contains any data beyond the data retention days.

After make sure that purging has been happened as expected, but yet event store DB size growing is observed. Well..what can we do now..?

We can suspect that this could be happening because of an indexing issue. If you go to event store database and check the tables, you will see tables begin with their names either anx_7 or anx_8. If you observe the sizes of these tables and if the sizes of anx_8 tables are relatively and considerably larger than anx_7 table, then we can suspect this is because of an indexing issue.

Then the solution would be re-indexing the database.

Following steps will help you to re-index the DB of APIM Anlaytics server.

1. Shutdown the server.
2. Drop the tables starts with anx_8 in event store database.
3. Delete <ANALYTICS_HOME>/repository/data directory
4. Open <APIM-ANALYTICS_HOME>/repository/conf/analytics/local-shard-allocation-config.conf and you will see a few shard entries(depending on your deployment) as {number},NORMAL.
5. Change its state from NORMAL to INIT

{NUMBER},NORMAL →{NUMBER},INIT

6. Restart the server

You do not have to worry about any significant data loss when re-indexing is performed. It only drops the tables(anx_8 tables) related to event metadata and these tables will be created again once the re-indexing is performed.

If all the above steps are followed, you can now check the sizes of tables in event store DB. If re-indexing has been performed as expected, then you would not see the considerable size difference between anx_7 and anx_8 tables.

Well..now you are aware of the steps to be taken if you observe APIM Analytics DB is growing. Hope you got an idea of re-indexing APIM Analytics event store DB to solve your issue.

Enjoy reading….!