How to purge data in APIM Analytics
This article will be helpful for readers who are aware of what is WSO2, WSO2 API Manager and WSO2 APIM Analytics are. Further this article will affect to APIM Analytics version 2.0 and 2.1. (2.x products)
When you configure WSO2 API Manager with WSO2 APIM Analytics, APIM will be publishing all its events to analytics server.
With the events getting published to APIM Analytics server, event tables will be growing. Hence it is recommended to purge data of the following tables, keeping data only within a certain period of time. Any data beyond that will get purged.
- ORG_WSO2_APIMGT_STATISTICS_WORKFLOW
- ORG_WSO2_APIMGT_STATISTICS_PERMINUTEREQUEST
- ORG_WSO2_APIMGT_STATISTICS_PERHOURREQUEST
- ORG_WSO2_APIMGT_STATISTICS_PERDAYREQUEST
- ORG_WSO2_APIMGT_STATISTICS_PERMINUTERESPONSE
- ORG_WSO2_APIMGT_STATISTICS_PERHOURRESPONSE
- ORG_WSO2_APIMGT_STATISTICS_PERDAYRESPONSE
- ORG_WSO2_APIMGT_STATISTICS_PERMINUTEEXECUTIONTIMES
- ORG_WSO2_APIMGT_STATISTICS_PERHOUREXECUTIONTIMES
- ORG_WSO2_APIMGT_STATISTICS_PERDAYEXECUTIONTIMES
- ORG_WSO2_APIMGT_STATISTICS_THROTTLE
- ORG_WSO2_APIMGT_STATISTICS_FAULT
- ORG_WSO2_CARBON_IDENTITY_OAUTH_TOKEN_ISSUANCE
- LOGANALYZER
In APIM Analytics you can achieve data purging in two ways.
Global data purging method
Only a single file has to be edited.
Open the file,
<analytics_home>/repository/conf/analytics/analytics-config.xml
By default purging is disabled. You can check it when you open the above file and navigate to data purging related configurations and it will look like below.
Let me explain what each field within this configuration means.
- purging-enable
- This property will stand to indicate if the purging is enabled or disabled. To enable purging, you will have to set this property to true.
2. cron-expression
- This property stands for scheduling a time for purging tables. You can provide a time as a cron string and the purging will happen as you defined.
3. purge-include-tables
- Tables you defined here will be only subjected to purging. It is highly recommend to purge only the above mentioned tables.
4. data-retention-days
- This property allows you to define the time period of the data to keep in the tables. Any data older than this time period will be removed from the tables.
Sample configuration file is shown below. Here I have enabled purging for above tables and I have defined the data retention days as 7 days. Which means the defined tables will not have data older than 7 days. Purging will happen every day 00:00:00 time.
Through APIM Analytics management console
After you start the APIM Analytics server, you can go to the management console using {host}:9444 . For example localhost:9444.
You will have to follow Home> Manage > Interactive Analytics > Data Explorer path.
Then you will be presented with following window and you can choose the tables to purge. Keep in mind it is only recommended to purge the tables mentioned above. Otherwise you will end up loosing summarized useful data.
Here I have selected ORG_WSO2_APIMGT_STATISTICS_PERMINUTERESPONSE table.
Click on Schedule Data Purging button. Then the following window will pop up. There you can define the data retention period and schedule a time for purging.
Click save and the data will be purged in the selected tables on scheduled time.
Voilà…!!..Now the data in the per minute response table will be purged and it would not have any data older than 7 days and the purging will be executed everyday 00:00 time.
Well..if you could get through the whole article, you are now aware of purging data in APIM Analytics :) .
I hope you understood everything clear and the article is helpful for those who were not aware of this before. Enjoy reading…!!
