Sometimes we end up with phantom messages in ActiveMq or Artemis engines.
This is usually the case when due to some network issue/ database issue and inconsistent commit was done.
I have no idea if this is a bug of the engine or a known case that can happen because of external causes.
The issue with the “phantom messages” manifests like this. The counter associated with a queue has a diferent value than the actual number of messages from that queue.
The bellow example shows a case where the counter reported that amessage is still there but in fact the queue is empty. This is a problem when a client application is using the counter API call to estimate if all the queued message were processed.
List message reports no message:
[standalone@localhost:9992 /] /subsystem=messaging-activemq/server=default/jms-queue=myQueue:list-messages
{
"outcome" => "success",
"result" => []
}
Count message reports 1 message:
[standalone@localhost:9992 /] /subsystem=messaging-activemq/server=default/jms-queue=myQueue:count-messages
{
"outcome" => "success",
"result" => 1L
}
Solution:
Step 1:
Stop any consumer or producer applications. The easiest way is to simply undeploy the application.
[standalone@localhost:9992 /] /deployment=my_app.ear:undeploy
{
"outcome" => "success",
"result" => 1L
}
Step 2:
Reset the counter:
[standalone@localhost:9992 /] /subsystem=messaging-activemq/server=default/jms-queue=accountingSpace:reset-message-counter
Step 3:
Start consumer or producer applications
[standalone@localhost:9992 /] /deployment=my_app.ear:deploy
{
"outcome" => "success",
"result" => 1L
}
Interesting enough that there is an API call for counter reset. That means that the possibility of a corrupt counter is somehow accounted for in the API design.