{"id":8948,"date":"2019-04-15T14:10:24","date_gmt":"2019-04-15T17:10:24","guid":{"rendered":"http:\/\/blog.plataformatec.com.br\/?p=8948"},"modified":"2019-04-15T14:44:26","modified_gmt":"2019-04-15T17:44:26","slug":"announcing-the-rabbitmq-connector-for-broadway","status":"publish","type":"post","link":"http:\/\/blog.plataformatec.com.br\/2019\/04\/announcing-the-rabbitmq-connector-for-broadway\/","title":{"rendered":"Announcing the RabbitMQ Connector for Broadway"},"content":{"rendered":"<p>Since the <a href=\"http:\/\/blog.plataformatec.com.br\/2019\/02\/announcing-broadway\/\" target=\"_blank\" rel=\"noopener noreferrer\">announcement of Broadway<\/a> a couple of months ago, we&#8217;ve received a very positive response from the community. We were able to gather some great feedback regarding the architecture, suggestions for extending the API and requests for new connectors to allow attaching other technologies to Broadway pipelines. One of the most requested additions was a RabbitMQ connector that would allow Broadway pipelines to consume messages from RabbitMQ.<\/p>\n<p>Today we&#8217;re glad to announce the first release of <a href=\"https:\/\/github.com\/plataformatec\/broadway_rabbitmq\/\" target=\"_blank\" rel=\"noopener noreferrer\">BroadwayRabbitMQ<\/a>, a RabbitMQ connector for Broadway by <a href=\"https:\/\/plataformatec.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Plataformatec<\/a>.<\/p>\n<h2>Features<\/h2>\n<p>The BroadwayRabbitMQ connector ships with a custom Broadway producer that is responsible for:<\/p>\n<ul>\n<li>Consuming messages from a RabbitMQ queue<\/li>\n<li>Automatically acknowledges\/rejects messages<\/li>\n<li>Handles connection outages using backoff for retries<\/li>\n<\/ul>\n<h2>Usage<\/h2>\n<p>In order to consume messages from RabbitMQ, you need to configure your Broadway pipeline with one or more <code>BroadwayRabbitMQ<\/code> producers:<\/p>\n<pre><code class=\"elixir\">defmodule MyBroadway do\n   use Broadway\n\n   def start_link(_opts) do\n     Broadway.start_link(__MODULE__,\n       name: __MODULE__,\n       producers: [\n         default: [\n           module: {BroadwayRabbitMQ.Producer,\n             queue: \"my_queue\"\n           },\n           stages: 2\n         ]\n       ],\n       processors: [\n         default: [\n           stages: 50\n         ]\n       ]\n     )\n   end\n\n   def handle_message(_, message, _) do\n     IO.inspect(message.data, label: \"Got message\")\n     message\n   end\n end\n<\/code><\/pre>\n<p>If you&#8217;re consuming data from an existing broker that requires authorization, you&#8217;ll need to provide your credentials using the <code>:connection<\/code> option:<\/p>\n<pre><code class=\"elixir\">...\nproducers: [\n  default: [\n    module: {BroadwayRabbitMQ.Producer,\n      queue: \"my_queue\",\n      connection: [\n        username: \"user\",\n        password: \"password\",\n      ]\n      ...\n    }\n  ]\n]\n...\n<\/code><\/pre>\n<p>The producer also exposes other options to customize <code>:connection<\/code> and <code>:qos<\/code> configurations. For a full list of options, please see the <a href=\"https:\/\/hexdocs.pm\/broadway_rabbitmq\/BroadwayRabbitMQ.Producer.html\" target=\"_blank\" rel=\"noopener noreferrer\">BroadwayRabbitMQ.Producer<\/a> documentation.<\/p>\n<h2>Next steps<\/h2>\n<p>If you want to give <code>BroadwayRabbitMQ<\/code> a try, we strongly recommend you to take a look at our <a href=\"https:\/\/hexdocs.pm\/broadway\/rabbitmq.html\" target=\"_blank\" rel=\"noopener noreferrer\">RabbitMQ Guide<\/a>. There you&#8217;ll find more details on usage and configuration, as well as valuable information about some of the design choices behind the implementation.<\/p>\n<p>We&#8217;ll continue to collect feedback from the community in order to improve the tools around Broadway. We also encourage you to contribute to this project or any other in the Broadway ecosystem, either by reporting bugs or suggesting improvements. If you&#8217;re working with data ingestion\/processing pipelines and you&#8217;d like to discuss GenStage, Broadway or anything Elixir related, <a href=\"http:\/\/plataformatec.com.br\/contact\" target=\"_blank\" rel=\"noopener noreferrer\">reach out<\/a> and we&#8217;ll be glad to talk.<\/p>\n<p>Happy coding!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Since the announcement of Broadway a couple of months ago, we&#8217;ve received a very positive response from the community. We were able to gather some great feedback regarding the architecture, suggestions for extending the API and requests for new connectors to allow attaching other technologies to Broadway pipelines. One of the most requested additions was &#8230; <a class=\"read-more-link\" href=\"http:\/\/blog.plataformatec.com.br\/2019\/04\/announcing-the-rabbitmq-connector-for-broadway\/\">\u00bb<\/a><\/p>\n","protected":false},"author":77,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[1],"tags":[143],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts\/8948"}],"collection":[{"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/users\/77"}],"replies":[{"embeddable":true,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/comments?post=8948"}],"version-history":[{"count":10,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts\/8948\/revisions"}],"predecessor-version":[{"id":8960,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts\/8948\/revisions\/8960"}],"wp:attachment":[{"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/media?parent=8948"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/categories?post=8948"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/tags?post=8948"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}