During the SAS Viya upgrade process, pods can be removed, renamed, or consolidated. These pod changes are documented in the SAS Viya Platform Operations: What's New... section.  If these changed pods used RabbitMQ queues, the queues with the old names still exist after the upgrade. After five days of the queues not being used, the sas-rabbitmq-server-0 pod deletes the unused queues.   

Until the queues are deleted, messages are received by the unused queue but are not consumed. As a result, memory and disk usage of RabbitMQ can increase.  

Resolution

A Kubernetes admin with exec access can manually delete the queues with no consumers and unread messages before they are automatically deleted.

1. Use the following command to list the queues that have no consumers, excluding dead letter and retry queues:
   
   kubectl -n <NS> exec sas-rabbitmq-server-0 -c sas-rabbitmq-server -it -- bash -c 'source /rabbitmq/data/.bashrc;/opt/sas/viya/home/lib/rabbitmq-server/sbin/rabbitmqctl list_queues name messages_ready consumers | grep 0$' | grep -v \.dlq | grep -v retry\.*s
   
   For the list of queues returned, the first number is the number of messages, and the second number is the number of consumers. Here is an example:
   
   sas.application.decisions.com-sas-dcm-decisions-code-file-events-DecisionCodeFileEventListener.quorum 332871 0 
   
   
2. For each unused queue, use the following command to delete the queue:
   
   kubectl -n <NS> exec sas-rabbitmq-server-0 -c sas-rabbitmq-server -it -- bash -c ' \
   QUEUE_TO_DELETE=<queue name with non-zero messages and 0 consumers>; \
   source /rabbitmq/data/.bashrc; \
   /opt/sas/viya/home/lib/rabbitmq-server/sbin/rabbitmqctl delete_queue $QUEUE_TO_DELETE --if-unused'