{"id":8250,"date":"2019-01-04T11:46:17","date_gmt":"2019-01-04T13:46:17","guid":{"rendered":"http:\/\/blog.plataformatec.com.br\/?p=8250"},"modified":"2019-01-04T18:19:01","modified_gmt":"2019-01-04T20:19:01","slug":"building-a-new-mysql-adapter-for-ecto-part-iv-ecto-integration","status":"publish","type":"post","link":"http:\/\/blog.plataformatec.com.br\/2019\/01\/building-a-new-mysql-adapter-for-ecto-part-iv-ecto-integration\/","title":{"rendered":"Building a new MySQL adapter for Ecto Part IV: Ecto Integration"},"content":{"rendered":"
Welcome to the “Building a new MySQL adapter for Ecto” series:<\/p>\n
After DBConnection<\/a> integration we have a driver that should be usable on its own. The next step is to integrate it with Ecto so that we can:<\/p>\n If you ever worked with Ecto, you’ve seen code like:<\/p>\n The Adapters are required to implement at least There’s also a separate Ecto SQL<\/a> project which ships with its own set of adapter specifications on top of the ones from Ecto. Conveniently, it also includes a Let’s try using the When we compile it, we’ll get a bunch of warnings as we haven’t implemented any of the callbacks yet.<\/p>\n Notably, we get a Since we’ve leveraged DBConnection in the MyXQL driver, these functions are simply delegating to driver. Let’s implement something a little bit more interesting.<\/p>\n Did you ever wonder how Let’s see how the constraint violation error looks exactly:<\/p>\n MySQL responds with error code Ok, let’s finally implement the callback:<\/p>\n Let’s break this down. We expect that the driver raises an exception struct on constraint violation, we then match on the particular error code, extract the field name from the error message, and return that as keywords list.<\/p>\n (To make this more understandable, in the MyXQL<\/a> project we’ve added error code\/name mapping so we pattern match like this instead: To get a feeling of what other subtle changes we may have between data stores, let’s implement one more callback, back in the While MySQL has a As you can see there might be quite a bit of discrepancies between adapters and data stores. For this reason, besides providing adapter specifications, Ecto ships with integration tests that can be re-used by adapter libraries.<\/p>\n Here’s a set of basic integration test cases and support files in Ecto, see: And here’s an example how a separate package might leverage these. Turns out that and has a few of its own<\/a>.<\/p>\n When implementing a 3rd-party SQL adapter for Ecto we already have a lot of integration tests to run against!<\/p>\n In this article we have briefly looked at integrating our driver with Ecto and Ecto SQL.<\/p>\n Ecto helps with the integration by providing:<\/p>\n We’re also concluding our adapter series. Some of the overarching themes were:<\/p>\n As this article is being published, we’re getting closer to shipping MyXQL’s first release as well as making it the default MySQL adapter in upcoming Ecto v3.1. You can see the progress on elixir-ecto\/ecto_sql#66<\/a>.<\/p>\n Happy coding!<\/p>\n <\/a><\/p>\n","protected":false},"excerpt":{"rendered":" Welcome to the “Building a new MySQL adapter for Ecto” series: Part I: Hello World Part II: Encoding\/Decoding Part III: DBConnection Integration Part IV: Ecto Integration (you’re here!) After DBConnection integration we have a driver that should be usable on its own. The next step is to integrate it with Ecto so that we can: … \u00bb<\/a><\/p>\n","protected":false},"author":70,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[1],"tags":[238,143],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts\/8250"}],"collection":[{"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/users\/70"}],"replies":[{"embeddable":true,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/comments?post=8250"}],"version-history":[{"count":31,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts\/8250\/revisions"}],"predecessor-version":[{"id":8338,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/posts\/8250\/revisions\/8338"}],"wp:attachment":[{"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/media?parent=8250"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/categories?post=8250"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/blog.plataformatec.com.br\/wp-json\/wp\/v2\/tags?post=8250"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
mix ecto.create<\/code><\/a> and
mix ecto.migrate<\/code><\/a>, and finally using Ecto SQL Sandbox<\/a> to manage clean slate between tests<\/li>\n
Ecto Adapter<\/h2>\n
defmodule MyApp.Repo do\n use Ecto.Repo,\n adapter: Ecto.Adapters.MySQL,\n otp_app: :my_app\nend\n<\/code><\/pre>\n
adapter<\/code> is a module that implements Ecto Adapter specifications:<\/p>\n
\n
Ecto.Adapter<\/code><\/a> – minimal API required from adapters<\/li>\n
Ecto.Adapter.Queryable<\/code><\/a> – plan, prepare, and execute queries leveraging query cache<\/li>\n
Ecto.Adapter.Schema<\/code><\/a> – insert, update, and delete structs as well as autogenerate IDs<\/li>\n
Ecto.Adapter.Storage<\/code><\/a> – storage API used by e.g.
mix ecto.create<\/code> and
mix ecto.drop<\/code><\/li>\n
Ecto.Adapter.Transaction<\/code><\/a> – transactions API<\/li>\n<\/ul>\n
Ecto.Adapter<\/code> behaviour. The remaining behaviours are optional as some data stores don’t support transactions or creating\/dropping the storage (e.g. some cloud services).<\/p>\n
Ecto.Adapters.SQL<\/code><\/a> module that we can use<\/em>, which implements most of the callbacks and lets us worry mostly about generating appropriate SQL.<\/p>\n
Ecto SQL Adapter<\/h2>\n
Ecto.Adapters.SQL<\/code> module:<\/p>\n
defmodule MyXQL.EctoAdapter do\n use Ecto.Adapters.SQL,\n driver: :myxql,\n migration_lock: \"FOR UPDATE\"\nend\n<\/code><\/pre>\n
warning: function supports_ddl_transaction?\/0 required by behaviour Ecto.Adapter.Migration is not implemented (in module MyXQL.EctoAdapter)\n lib\/a.ex:1\n\nwarning: function MyXQL.EctoAdapter.Connection.all\/1 is undefined (module MyXQL.EctoAdapter.Connection is not available)\n lib\/a.ex:2\n\nwarning: function MyXQL.EctoAdapter.Connection.delete\/4 is undefined (module MyXQL.EctoAdapter.Connection is not available)\n lib\/a.ex:2\n\n(...)\n<\/code><\/pre>\n
module MyXQL.EctoAdapter.Connection is not available<\/code> warning. The SQL adapter specification requires us to implement a separate connection module (see
Ecto.Adapters.SQL.Connection<\/code><\/a> behaviour) which will leverage, you guessed it, DBConnection. Let’s try that now and implement a couple of callbacks:<\/p>\n
defmodule MyXQL.EctoAdapter.Connection do\n @moduledoc false\n @behaviour Ecto.Adapters.SQL.Connection\n\n @impl true\n def child_spec(opts) do\n MyXQL.child_spec(opts)\n end\n\n @impl true\n def prepare_execute(conn, name, sql, params, opts) do\n MyXQL.prepare_execute(conn, name, sql, params, opts)\n end\nend\n<\/code><\/pre>\n
Ecto.Changeset.unique_constraint\/3<\/code><\/a> is able to transform a SQL constraint violation failure into a changeset error? Turns out that
unique_constriant\/3<\/code> keeps a mapping between unique key constraint name and fields these errors should be reported on. The code that makes it work is executed in the repo and the adapter when the structs are persisted. In particular, the adapter should implement the
Ecto.Adapters.SQL.Connection.to_constraints\/1<\/code><\/a> callback. Let’s take a look:<\/p>\n
iex> b Ecto.Adapters.SQL.Connection.to_constraints\n@callback to_constraints(exception :: Exception.t()) :: Keyword.t()\n\nReceives the exception returned by c:query\/4.\n\nThe constraints are in the keyword list and must return the constraint type,\nlike :unique, and the constraint name as a string, for example:\n\n [unique: \"posts_title_index\"]\n\nMust return an empty list if the error does not come from any constraint.\n<\/code><\/pre>\n
$ mysql -u root myxql_test\nmysql> CREATE TABLE uniques (x INTEGER UNIQUE);\nQuery OK, 0 rows affected (0.17 sec)\n\nmysql> INSERT INTO uniques VALUES (1);\nQuery OK, 1 row affected (0.08 sec)\n\nmysql> INSERT INTO uniques VALUES (1);\nERROR 1062 (23000): Duplicate entry '1' for key 'x'\n<\/code><\/pre>\n
1062<\/code>. We can further look into the error by using
perror<\/code>
\ncommand-line utility that ships with MySQL installation:<\/p>\n% perror 1062\nMySQL error code 1062 (ER_DUP_ENTRY): Duplicate entry '%-.192s' for key %d\n<\/code><\/pre>\n
defmodule MyXQL.EctoAdapter.Connection do\n # ...\n\n @impl true\n def to_constraints(%MyXQL.Error{mysql: %{code: 1062}, message: message}) do\n case :binary.split(message, \" for key \") do\n [_, quoted] -> [unique: strip_quotes(quoted)]\n _ -> []\n end\n end\nend\n<\/code><\/pre>\n
mysql: %{code: :ER_DUP_ENTRY}<\/code>.)<\/p>\n
MyXQL.EctoAdapter<\/code> module.<\/p>\n
BOOLEAN<\/code> type, turns out it’s simply an alias to
TINYINT<\/code> and its possible values are
1<\/code> and
0<\/code>. These sort of discrepancies are handled by the
dumpers\/2<\/code><\/a> and
loaders\/2<\/code><\/a> callbacks, let’s implement the latter:<\/p>\n
defmodule MyXQL.EctoAdapter do\n # ...\n\n @impl true\n def loaders(:boolean, type), do: [&bool_decode\/1, type]\n # ...\n def loaders(_, type), do: [type]\n\n defp bool_decode(<<0>>), do: {:ok, false}\n defp bool_decode(<<1>>), do: {:ok, true}\n defp bool_decode(0), do: {:ok, false}\n defp bool_decode(1), do: {:ok, true}\n defp bool_decode(other), do: {:ok, other}\nend\n<\/code><\/pre>\n
Integration Tests<\/h2>\n
.\/integration_test\/<\/code><\/a> directory.<\/p>\n
ecto_sql<\/code> uses
ecto<\/code> integration tests:<\/p>\n
# ecto_sql\/integration_test\/mysql\/all_test.exs\necto = Mix.Project.deps_paths[:ecto]\nCode.require_file \"#{ecto}\/integration_test\/cases\/assoc.exs\", __DIR__\nCode.require_file \"#{ecto}\/integration_test\/cases\/interval.exs\", __DIR__\n# ...\n<\/code><\/pre>\n
Conclusion<\/h2>\n
\n
Ecto.Adapters.SQL<\/code> module that we can use<\/em> to build adapters for relational databases even faster<\/li>\n
\n