Discussion about the future of fedmsg.


Because of my work on Google Summer of Code this year, I was invited to attend to the Fedora Contributor Conference (Flock) as volunteer, helping the organization staff to record some sessions and writing about what was discussed on then. This year, the FLock Conference was in Cape Cod, Massachusetts. Was a incredible experience, allowing me to keep up with great discussions made by the Fedora developers. On this post I will make a resume of what was discussed on the session The Future of fedmsg?, proposed by Jeremy Cline.

The Session

The Fedora infrastructure have several different services that need to talk to each other. One simple example is the AutoQA service, that listen to some events triggered by the fedpkg library. If we have only two services interacting the problem is minimal, but when several applications request and response to other several applications, the problem becomes huge. FEDerated MeSsaGe bus (fedmsg), is a python package and API defining a brokerless messaging architecture to send and receive messages to and from applications.

fedmsg does not have a broker to manage the publish/subscribe process, done by the services that interacts with it. This leads to a problem of performance and reliability, because every service that consumes a message from fedmsg have to subscribe to every existent topic. Other issue with the absence of a broker, is that it is common to lose messages, that not always get to the destination. Jeremy proposed to use a broker (or several brokers) to fix these issues, and made a demo code showing the benefits of using a broker, instead of the current architecture of fedmsg.

A great discussion emerged from this demo code, including the reflection if Fedora really needs that fedmsg be reliable. Other problem pointed by Jeremy was the documentation of fedmsg, and the existent tools to consume and publish messages (fedmsg-hub have a setup that is quite confuse). This was my review of the session, and based on my work on Google Summer of Code (I had used fedmsg to consume the Anitya events), I agree with Jeremy. Adding a broker to manage the publish/subscribe process, could improve the consume of resources of fedmsg, would facilitate to add new services consuming messages from the API and would make fedmsg more reliable.