<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="https://janko.io/feed.xml" rel="self" type="application/atom+xml" /><link href="https://janko.io/" rel="alternate" type="text/html" /><updated>2026-05-27T11:23:10+00:00</updated><id>https://janko.io/feed.xml</id><title type="html">Janko Marohnić</title><author><name>Janko Marohnić</name><email>janko@hey.com</email></author><entry><title type="html">Extending Ruby LSP with Prism</title><link href="https://janko.io/extending-ruby-lsp-with-prism/" rel="alternate" type="text/html" title="Extending Ruby LSP with Prism" /><published>2026-05-26T00:00:00+00:00</published><updated>2026-05-26T00:00:00+00:00</updated><id>https://janko.io/extending-ruby-lsp-with-prism</id><content type="html" xml:base="https://janko.io/extending-ruby-lsp-with-prism/"><![CDATA[<p><a href="https://shopify.github.io/ruby-lsp/">Ruby LSP</a> is a wonderful language server built on top of Prism, <a href="https://github.com/shopify/rubydex">Rubydex</a> and RBS. It implements a <a href="https://shopify.github.io/ruby-lsp/#general-features">variety of features</a> that enrich the code editing experience in Ruby projects. Its <a href="https://shopify.github.io/ruby-lsp/add-ons.html">add-on</a> architecture allows extending it with <a href="https://shopify.github.io/ruby-lsp/rails-add-on">Rails features</a>, <a href="https://github.com/rubocop/rubocop/blob/master/lib/ruby_lsp/rubocop/addon.rb">Rubocop support</a> and custom functionality.</p>

<p>Coming from Vim, I was really used to <a href="https://github.com/tpope/vim-rails">rails.vim</a>. When I switched to Zed, I started using Ruby LSP. In some ways I felt like I’d gained superpowers, as now I had all these modern editor features that are possible because my Ruby code is actually being parsed. On the other hand, I found there were some features I was missing.</p>

<p>One such feature was following <code class="language-plaintext highlighter-rouge">render</code> calls in view templates. Rails.vim offered a <code class="language-plaintext highlighter-rouge">gf</code> (“go to file”) mapping, which when hovering over a <code class="language-plaintext highlighter-rouge">render</code> call would take me to the partial being rendered. In LSP terminology this functionality is called “go to definition”. If I were to implement it, I knew it had to live in the Rails add-on.</p>

<h2 id="lsp-mechanics">LSP mechanics</h2>

<p>Before we can get into the weeds, we need to establish how language servers work. The Language Server Protocol (LSP) defines JSON-RPC messaging between a code editor and a server process.</p>

<p>When you hover over a Ruby class constant and hold <code class="language-plaintext highlighter-rouge">cmd</code>, Zed will send a message to Ruby LSP in the following format:</p>

<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
  </span><span class="nl">"jsonrpc"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2.0"</span><span class="p">,</span><span class="w">
  </span><span class="nl">"id"</span><span class="p">:</span><span class="w"> </span><span class="mi">5</span><span class="p">,</span><span class="w">
  </span><span class="nl">"method"</span><span class="p">:</span><span class="w"> </span><span class="s2">"textDocument/definition"</span><span class="p">,</span><span class="w">
  </span><span class="nl">"params"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
    </span><span class="nl">"textDocument"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
      </span><span class="nl">"uri"</span><span class="p">:</span><span class="w"> </span><span class="s2">"file:///path/to/source.rb"</span><span class="w">
    </span><span class="p">},</span><span class="w">
    </span><span class="nl">"position"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nl">"line"</span><span class="p">:</span><span class="w"> </span><span class="mi">7</span><span class="p">,</span><span class="w"> </span><span class="nl">"character"</span><span class="p">:</span><span class="w"> </span><span class="mi">27</span><span class="w"> </span><span class="p">}</span><span class="w">
  </span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>

<p>Since Ruby LSP indexes all constant declarations on initialization, it can then respond with the location where the constant is defined:</p>

<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
  </span><span class="nl">"id"</span><span class="p">:</span><span class="w"> </span><span class="mi">5</span><span class="p">,</span><span class="w">
  </span><span class="nl">"result"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
    </span><span class="p">{</span><span class="w">
      </span><span class="nl">"targetUri"</span><span class="p">:</span><span class="w"> </span><span class="s2">"file:///path/to/class.rb"</span><span class="p">,</span><span class="w">
      </span><span class="nl">"targetRange"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
        </span><span class="nl">"start"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nl">"line"</span><span class="p">:</span><span class="w"> </span><span class="mi">8</span><span class="p">,</span><span class="w"> </span><span class="nl">"character"</span><span class="p">:</span><span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="p">},</span><span class="w">
        </span><span class="nl">"end"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nl">"line"</span><span class="p">:</span><span class="w"> </span><span class="mi">253</span><span class="p">,</span><span class="w"> </span><span class="nl">"character"</span><span class="p">:</span><span class="w"> </span><span class="mi">5</span><span class="w"> </span><span class="p">}</span><span class="w">
      </span><span class="p">},</span><span class="w">
      </span><span class="nl">"targetSelectionRange"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
        </span><span class="nl">"start"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nl">"line"</span><span class="p">:</span><span class="w"> </span><span class="mi">8</span><span class="p">,</span><span class="w"> </span><span class="nl">"character"</span><span class="p">:</span><span class="w"> </span><span class="mi">9</span><span class="w"> </span><span class="p">},</span><span class="w">
        </span><span class="nl">"end"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="nl">"line"</span><span class="p">:</span><span class="w"> </span><span class="mi">8</span><span class="p">,</span><span class="w"> </span><span class="nl">"character"</span><span class="p">:</span><span class="w"> </span><span class="mi">19</span><span class="w"> </span><span class="p">}</span><span class="w">
      </span><span class="p">}</span><span class="w">
    </span><span class="p">}</span><span class="w">
  </span><span class="p">],</span><span class="w">
  </span><span class="nl">"jsonrpc"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2.0"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>

<p>Here we can see Ruby LSP returned the location of the <code class="language-plaintext highlighter-rouge">class ... end</code> block (<code class="language-plaintext highlighter-rouge">targetRange</code>) as well as the constant name the editor should select (<code class="language-plaintext highlighter-rouge">targetSelectRange</code>). Notice that the <code class="language-plaintext highlighter-rouge">result</code> field is an array, allowing for multiple locations in case the class is re-opened more than once (for which Zed will open a <a href="https://zed.dev/docs/multibuffers">multibuffer</a>).</p>

<h2 id="custom-add-on">Custom add-on</h2>

<p>Back to the task at hand. I needed to test out my Ruby LSP extension locally first. It turns out Ruby LSP will automatically pick up any add-ons in your project directory, they just need to match <code class="language-plaintext highlighter-rouge">**/ruby_lsp/**/addon.rb</code>. So, I put mine in <code class="language-plaintext highlighter-rouge">lib/ruby_lsp/my_app/addon.rb</code>:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># lib/ruby_lsp/my_app/addon.rb</span>
<span class="nb">require</span> <span class="s2">"ruby_lsp/addon"</span>

<span class="k">module</span> <span class="nn">RubyLsp</span>
  <span class="k">module</span> <span class="nn">MyApp</span>
    <span class="k">class</span> <span class="nc">Addon</span> <span class="o">&lt;</span> <span class="no">RubyLsp</span><span class="o">::</span><span class="no">Addon</span>
      <span class="k">def</span> <span class="nf">activate</span><span class="p">(</span><span class="n">global_state</span><span class="p">,</span> <span class="n">outgoing_queue</span><span class="p">)</span>
        <span class="n">outgoing_queue</span> <span class="o">&lt;&lt;</span> <span class="no">Notification</span><span class="p">.</span><span class="nf">window_log_message</span><span class="p">(</span><span class="s2">"Activated My App addon"</span><span class="p">)</span>
      <span class="k">end</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>If everything works correctly, after restarting your Ruby LSP server, you should see the “Activated My App addon” message in the language server logs.</p>

<p>When Ruby LSP receives a <code class="language-plaintext highlighter-rouge">textDocument/definition</code> request, it calls <code class="language-plaintext highlighter-rouge">#create_definition_listener</code> on every add-on with some parameters, allowing them to add their own locations to the response. Let’s override it:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># lib/ruby_lsp/my_app/addon.rb</span>
<span class="c1"># ...</span>
<span class="nb">require_relative</span> <span class="s2">"definition"</span>

<span class="k">module</span> <span class="nn">RubyLsp</span>
  <span class="k">module</span> <span class="nn">MyApp</span>
    <span class="k">class</span> <span class="nc">Addon</span> <span class="o">&lt;</span> <span class="no">RubyLsp</span><span class="o">::</span><span class="no">Addon</span>
      <span class="c1"># ...</span>
      <span class="k">def</span> <span class="nf">create_definition_listener</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
        <span class="no">Definition</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
      <span class="k">end</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># lib/ruby_lsp/my_app/definition.rb</span>
<span class="k">module</span> <span class="nn">RubyLsp</span>
  <span class="k">module</span> <span class="nn">MyApp</span>
    <span class="k">class</span> <span class="nc">Definition</span>
      <span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">response_builder</span><span class="p">,</span> <span class="n">uri</span><span class="p">,</span> <span class="n">node_context</span><span class="p">,</span> <span class="n">dispatcher</span><span class="p">)</span>
        <span class="vi">@response_builder</span> <span class="o">=</span> <span class="n">response_builder</span>
        <span class="vi">@path</span> <span class="o">=</span> <span class="n">uri</span><span class="p">.</span><span class="nf">to_standardized_path</span>
        <span class="vi">@node_context</span> <span class="o">=</span> <span class="n">node_context</span>
        <span class="vi">@dispatcher</span> <span class="o">=</span> <span class="n">dispatcher</span>
      <span class="k">end</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<h2 id="prism-drilling">Prism drilling</h2>

<p>Ruby LSP will use Prism to parse the source document where we activated go-to-definition, set up a <a href="https://ruby.github.io/prism/rb/Prism/Dispatcher.html">dispatcher</a> for walking the AST, and save the context of the AST node you’re hovering over. In our case, we want to react on partial names passed to <code class="language-plaintext highlighter-rouge">render</code> calls. Since these are strings, let’s register a listener for entering string nodes:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># lib/ruby_lsp/my_app/definition.rb</span>
<span class="k">module</span> <span class="nn">RubyLsp</span>
  <span class="k">module</span> <span class="nn">MyApp</span>
    <span class="k">class</span> <span class="nc">Definition</span>
      <span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">response_builder</span><span class="p">,</span> <span class="n">uri</span><span class="p">,</span> <span class="n">node_context</span><span class="p">,</span> <span class="n">dispatcher</span><span class="p">)</span>
        <span class="c1"># ...</span>
        <span class="n">dispatcher</span><span class="p">.</span><span class="nf">register</span><span class="p">(</span><span class="nb">self</span><span class="p">,</span> <span class="ss">:on_string_node_enter</span><span class="p">)</span>
      <span class="k">end</span>

      <span class="k">def</span> <span class="nf">on_string_node_enter</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
        <span class="c1"># ...</span>
      <span class="k">end</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>First things first, we can only support following <code class="language-plaintext highlighter-rouge">render</code> calls inside HTML+ERB templates, as in helpers we don’t have the view/controller context. So, let’s early return otherwise:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">on_string_node_enter</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
  <span class="k">return</span> <span class="k">unless</span> <span class="n">html_erb?</span>
<span class="k">end</span>

<span class="kp">private</span>

<span class="k">def</span> <span class="nf">html_erb?</span>
  <span class="vi">@path</span><span class="o">&amp;</span><span class="p">.</span><span class="nf">match?</span><span class="p">(</span><span class="sr">/\.html(\+\w+)?\.erb/</span><span class="p">)</span> <span class="c1"># handle template variants</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Unlike Solargraph, Ruby LSP has <a href="https://shopify.github.io/ruby-lsp/#erb-support">ERB support</a> that can extract Ruby code, allowing it to provide the same features as for regular Ruby files. Templating languages like Slim and Haml have much more complex grammars, making it difficult to know where Ruby code is, so as of this writing they’re not supported.</p>

<p>We’ll return if the string node is not a “partial argument”, which we’ll define shortly after:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">on_string_node_enter</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
  <span class="c1"># ...</span>
  <span class="k">return</span> <span class="k">unless</span> <span class="n">partial_argument?</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
<span class="k">end</span>

<span class="kp">private</span>

<span class="k">def</span> <span class="nf">partial_argument?</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
  <span class="c1"># ...</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Inside <code class="language-plaintext highlighter-rouge">#partial_argument?</code>, let’s now grab the call node from the node context. For <code class="language-plaintext highlighter-rouge">render "partial"</code> code, that would be the <code class="language-plaintext highlighter-rouge">render</code> call, which is what we care about. It’s also possible there is no call node, in case a string node isn’t passed to any method invocation, so we need to handle that as well.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">partial_argument?</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
  <span class="n">call_node</span> <span class="o">=</span> <span class="vi">@node_context</span><span class="p">.</span><span class="nf">call_node</span>
  <span class="k">return</span> <span class="k">unless</span> <span class="n">call_node</span>
  <span class="k">return</span> <span class="k">unless</span> <span class="n">call_node</span><span class="p">.</span><span class="nf">message</span> <span class="o">==</span> <span class="s2">"render"</span>
<span class="k">end</span>
</code></pre></div></div>

<p>We want to only accept <code class="language-plaintext highlighter-rouge">render</code> calls in template context (<code class="language-plaintext highlighter-rouge">self.render</code> technically counts here as well), so we reject any other receivers like <code class="language-plaintext highlighter-rouge">Foo.render</code>:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">partial_argument?</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
  <span class="c1"># ...</span>
  <span class="k">return</span> <span class="k">unless</span> <span class="n">call_node</span><span class="p">.</span><span class="nf">receiver</span><span class="p">.</span><span class="nf">nil?</span> <span class="o">||</span> <span class="n">call_node</span><span class="p">.</span><span class="nf">receiver</span><span class="p">.</span><span class="nf">is_a?</span><span class="p">(</span><span class="no">Prism</span><span class="o">::</span><span class="no">SelfNode</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>

<p>We’re interested whether the string node matches any of the <code class="language-plaintext highlighter-rouge">render</code> call arguments, so let’s retrieve them:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">partial_argument?</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
  <span class="c1"># ...</span>
  <span class="n">arguments</span> <span class="o">=</span> <span class="n">call_node</span><span class="p">.</span><span class="nf">arguments</span><span class="o">&amp;</span><span class="p">.</span><span class="nf">arguments</span> <span class="c1"># an array of arguments</span>
  <span class="k">return</span> <span class="k">unless</span> <span class="n">arguments</span>
<span class="k">end</span>
</code></pre></div></div>

<p>If our string node is the first positional argument of the <code class="language-plaintext highlighter-rouge">render</code> call (e.g. <code class="language-plaintext highlighter-rouge">render "foo"</code>), then we found our partial name:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">partial_argument?</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
  <span class="c1"># ...</span>
  <span class="k">return</span> <span class="kp">true</span> <span class="k">if</span> <span class="n">arguments</span><span class="p">.</span><span class="nf">first</span> <span class="o">==</span> <span class="n">node</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Otherwise we check for keyword arguments to handle explicit <code class="language-plaintext highlighter-rouge">render partial: "foo"</code> form as well. We only accept keyword arguments as the <em>first</em> argument, as we want to exclude <code class="language-plaintext highlighter-rouge">render "bar", partial: "foo"</code>, where <code class="language-plaintext highlighter-rouge">"foo"</code> would be the value of a <code class="language-plaintext highlighter-rouge">partial</code> local variable.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">partial_argument?</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
  <span class="c1"># ...</span>
  <span class="k">return</span> <span class="k">unless</span> <span class="n">arguments</span><span class="p">.</span><span class="nf">first</span><span class="p">.</span><span class="nf">is_a?</span><span class="p">(</span><span class="no">Prism</span><span class="o">::</span><span class="no">KeywordHashNode</span><span class="p">)</span>
  
  <span class="n">arguments</span><span class="p">.</span><span class="nf">first</span><span class="p">.</span><span class="nf">elements</span><span class="p">.</span><span class="nf">any?</span> <span class="k">do</span> <span class="o">|</span><span class="n">element</span><span class="o">|</span>
    <span class="k">next</span> <span class="k">unless</span> <span class="n">element</span><span class="p">.</span><span class="nf">is_a?</span><span class="p">(</span><span class="no">Prism</span><span class="o">::</span><span class="no">AssocNode</span><span class="p">)</span>
    <span class="k">next</span> <span class="k">unless</span> <span class="n">element</span><span class="p">.</span><span class="nf">key</span><span class="p">.</span><span class="nf">is_a?</span><span class="p">(</span><span class="no">Prism</span><span class="o">::</span><span class="no">SymbolNode</span><span class="p">)</span>

    <span class="n">element</span><span class="p">.</span><span class="nf">key</span><span class="p">.</span><span class="nf">value</span> <span class="o">==</span> <span class="s2">"partial"</span> <span class="o">&amp;&amp;</span> <span class="n">element</span><span class="p">.</span><span class="nf">value</span> <span class="o">==</span> <span class="n">node</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<h2 id="returning-locations">Returning locations</h2>

<p>Now that we’ve identified that our string node is indeed a partial name, let’s resolve the actual template file.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">on_string_node_enter</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
  <span class="c1"># ...</span>
  <span class="n">resolve_partial</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
<span class="k">end</span>

<span class="kp">private</span>

<span class="k">def</span> <span class="nf">resolve_partial</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
  <span class="c1"># ...</span>
<span class="k">end</span>
</code></pre></div></div>

<p>If the string contains a <code class="language-plaintext highlighter-rouge">/</code>, it holds an absolute partial path, otherwise it’s a relative partial name. Since template engines and formats can be mixed, we handle any file extension.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">resolve_partial</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
  <span class="n">partial_path</span> <span class="o">=</span> <span class="n">node</span><span class="p">.</span><span class="nf">content</span>

  <span class="n">template_glob</span> <span class="o">=</span> <span class="k">if</span> <span class="n">partial_path</span><span class="p">.</span><span class="nf">include?</span><span class="p">(</span><span class="s2">"/"</span><span class="p">)</span>
    <span class="o">*</span><span class="n">directory</span><span class="p">,</span> <span class="nb">name</span> <span class="o">=</span> <span class="n">partial_path</span><span class="p">.</span><span class="nf">split</span><span class="p">(</span><span class="s2">"/"</span><span class="p">)</span>
    <span class="no">File</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="s2">"app/views"</span><span class="p">,</span> <span class="o">*</span><span class="n">directory</span><span class="p">,</span> <span class="s2">"_</span><span class="si">#{</span><span class="nb">name</span><span class="si">}</span><span class="s2">.*"</span><span class="p">)</span>
  <span class="k">else</span>
    <span class="no">File</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="no">File</span><span class="p">.</span><span class="nf">dirname</span><span class="p">(</span><span class="vi">@path</span><span class="p">),</span> <span class="s2">"_</span><span class="si">#{</span><span class="n">partial_path</span><span class="si">}</span><span class="s2">.*"</span><span class="p">)</span>
  <span class="k">end</span>

  <span class="n">template_path</span> <span class="o">=</span> <span class="no">Dir</span><span class="p">[</span><span class="n">template_glob</span><span class="p">].</span><span class="nf">first</span>
  <span class="k">return</span> <span class="k">unless</span> <span class="n">template_path</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Finally, we return the partial path in the response:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">resolve_partial</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
  <span class="c1"># ...</span>
  <span class="vi">@response_builder</span> <span class="o">&lt;&lt;</span> <span class="no">Interface</span><span class="o">::</span><span class="no">Location</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span>
    <span class="ss">uri: </span><span class="no">URI</span><span class="o">::</span><span class="no">Generic</span><span class="p">.</span><span class="nf">from_path</span><span class="p">(</span><span class="ss">path: </span><span class="n">template_path</span><span class="p">).</span><span class="nf">to_s</span><span class="p">,</span>
    <span class="ss">range: </span><span class="no">Interface</span><span class="o">::</span><span class="no">Range</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span>
      <span class="ss">start: </span><span class="no">Interface</span><span class="o">::</span><span class="no">Position</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">line: </span><span class="mi">0</span><span class="p">,</span> <span class="ss">character: </span><span class="mi">0</span><span class="p">),</span>
      <span class="ss">end: </span><span class="no">Interface</span><span class="o">::</span><span class="no">Position</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">line: </span><span class="mi">0</span><span class="p">,</span> <span class="ss">character: </span><span class="mi">0</span><span class="p">)</span>
    <span class="p">)</span>
  <span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Once you restart Ruby LSP, you can now try it out in the editor:</p>

<p><img src="/images/go-to-definition.gif" alt="go to definition demo" /></p>

<h2 id="closing-words">Closing words</h2>

<p>I had no prior experience with Prism or any other Ruby parser, so going into this was scary at first. But working with Prism was surprisingly intuitive — I could see why it became the canonical Ruby parser. I didn’t expect to need to be so precise when inspecting node context, but it makes a lot of sense.</p>

<p>The template resolution shown was simplified for the article. It doesn’t handle controller inheritance, <code class="language-plaintext highlighter-rouge">:variants</code>, <code class="language-plaintext highlighter-rouge">:formats</code>, <code class="language-plaintext highlighter-rouge">:handlers</code>, alternative <code class="language-plaintext highlighter-rouge">view_paths</code> etc. For that it would need to call the actual controller to perform the view template lookup. I baked all this into my <a href="https://github.com/Shopify/ruby-lsp-rails/pull/659">pull request</a> to the Rails add-on, hopefully it will get merged :crossed_fingers:</p>

<p>I’m excited for all the improvements Shopify is doing for Ruby LSP, such as <a href="https://railsatscale.com/2026-05-12-one-engine-many-tools/">creating Rubydex</a> which makes code indexing efficient and comprehensive. It’s an important tool for making Ruby development feel modern, and the add-on architecture creates exciting possibilities.</p>]]></content><author><name>Janko Marohnić</name><email>janko@hey.com</email></author><summary type="html"><![CDATA[Ruby LSP is a wonderful language server built on top of Prism, Rubydex and RBS. It implements a variety of features that enrich the code editing experience in Ruby projects. Its add-on architecture allows extending it with Rails features, Rubocop support and custom functionality.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://janko.io/images/social.jpg" /><media:content medium="image" url="https://janko.io/images/social.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Passkey Authentication with Rodauth</title><link href="https://janko.io/passkey-authentication-with-rodauth/" rel="alternate" type="text/html" title="Passkey Authentication with Rodauth" /><published>2023-07-24T00:00:00+00:00</published><updated>2023-07-24T00:00:00+00:00</updated><id>https://janko.io/passkey-authentication-with-rodauth</id><content type="html" xml:base="https://janko.io/passkey-authentication-with-rodauth/"><![CDATA[<p><a href="https://developer.apple.com/passkeys/">Passkeys</a> are a modern alternative to passwords, where the user’s device performs the authentication, usually requiring some form of user verification (biometric identification, PIN). Passkeys are built on top of <a href="http://rodauth.jeremyevans.net/rdoc/files/doc/webauthn_rdoc.html">WebAuthn</a> specification, which is based on public-key cryptography. Keypairs are created for each website, and the public key is sent to the server, while the private key is securely stored on the device. This makes passkeys:</p>

<ul>
  <li>stronger than any password</li>
  <li>safe from data breaches</li>
  <li>safe from phishing attacks</li>
</ul>

<p>WebAuthn credentials are bound to the device that created them (think security keys like <a href="https://www.yubico.com/">YubiKey</a>), which is good for privacy since no company has your data, but losing your device could lock you out of your account. Passkeys add the ability to be backed up to the cloud and synchronized between multiple devices, which reduces the risk of passkeys getting lost.</p>

<p><a href="https://github.com/jeremyevans/rodauth">Rodauth</a> provides first class support for passkeys, implemented on top of the excellent <a href="https://github.com/cedarcode/webauthn-ruby">webauthn-ruby</a> gem. It enables using passkeys as a multifactor authentication method, or for passwordless login and registration. In addition to routes, views and database storage, it also provides the complete <a href="https://github.com/jeremyevans/rodauth/tree/master/javascript">JavaScript part</a> that interacts with <a href="https://developer.mozilla.org/en-US/docs/Web/API/Web_Authentication_API">Web Authentication API</a> for zero configuration.</p>

<p>In this article, I would like to show how to set each of these up in a Rails app that uses <a href="https://github.com/janko/rodauth-rails">rodauth-rails</a>. I’ll be using Safari on macOS Ventura, and have iCloud Keychain sync enabled, which is a requirement for Apple passkeys.</p>

<h2 id="multifactor-authentication">Multifactor authentication</h2>

<p>As I mentioned before, Rodauth supports registering passkeys as a multifactor authentication method, in addition to TOTP, recovery codes and SMS codes it already <a href="https://janko.io/adding-multifactor-authentication-in-rails-with-rodauth/">provides</a>.</p>

<p>We’ll start by creating the necessary database tables:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate rodauth:migration webauthn
<span class="nv">$ </span>rails db:migrate
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">CreateRodauthWebauthn</span> <span class="o">&lt;</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Migration</span>
  <span class="k">def</span> <span class="nf">change</span>
    <span class="c1"># stores WebAuthn user identifiers</span>
    <span class="n">create_table</span> <span class="ss">:account_webauthn_user_ids</span><span class="p">,</span> <span class="ss">id: </span><span class="kp">false</span> <span class="k">do</span> <span class="o">|</span><span class="n">t</span><span class="o">|</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">integer</span> <span class="ss">:id</span><span class="p">,</span> <span class="ss">primary_key: </span><span class="kp">true</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">foreign_key</span> <span class="ss">:accounts</span><span class="p">,</span> <span class="ss">column: :id</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">string</span> <span class="ss">:webauthn_id</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span>
    <span class="k">end</span>
    <span class="c1"># stores WebAuthn credentials</span>
    <span class="n">create_table</span> <span class="ss">:account_webauthn_keys</span><span class="p">,</span> <span class="ss">primary_key: </span><span class="p">[</span><span class="ss">:account_id</span><span class="p">,</span> <span class="ss">:webauthn_id</span><span class="p">]</span> <span class="k">do</span> <span class="o">|</span><span class="n">t</span><span class="o">|</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">references</span> <span class="ss">:account</span><span class="p">,</span> <span class="ss">foreign_key: </span><span class="kp">true</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">string</span> <span class="ss">:webauthn_id</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">string</span> <span class="ss">:public_key</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">integer</span> <span class="ss">:sign_count</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">datetime</span> <span class="ss">:last_use</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span><span class="p">,</span> <span class="ss">default: </span><span class="o">-&gt;</span> <span class="p">{</span> <span class="s2">"CURRENT_TIMESTAMP"</span> <span class="p">}</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Next, we’ll enable the <a href="http://rodauth.jeremyevans.net/rdoc/files/doc/webauthn_rdoc.html"><code class="language-plaintext highlighter-rouge">webauthn</code></a> feature in our Rodauth configuration:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="n">enable</span> <span class="ss">:webauthn</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>This will add routes for setting up, authenticating via and removing passkeys:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails rodauth:routes
<span class="c"># ...</span>
<span class="c"># GET/POST  /webauthn-auth    rodauth.webauthn_auth_path</span>
<span class="c"># GET/POST  /webauthn-setup   rodauth.webauthn_setup_path</span>
<span class="c"># GET/POST  /webauthn-remove  rodauth.webauthn_remove_path</span>
<span class="c"># ...</span>
</code></pre></div></div>

<p>Now when the user navigates to the page for managing multifactor authentication methods, they should see a link for setting up WebAuthn authentication.</p>

<p><img src="/images/rodauth-webauthn-setup-link.png" alt="Rodauth WebAuthn setup link" /></p>

<p>This page will show a button for registering a WebAuthn credential, which is already hooked up with the necessary JavaScript code, so clicking on it should show a native browser dialog for creating a new passkey. Once I verify biometric identification, my 2nd factor is set up.</p>

<p><img src="/images/rodauth-passkey-registration-dialog.png" alt="Rodauth passkey registration dialog" /></p>

<p>When this user is logging in the next time, once they’ve authenticated with 1st factor, they’re given the option to authenticate via a passkey for 2nd factor.</p>

<p><img src="/images/rodauth-webauthn-auth-link.png" alt="Rodauth WebAuthn auth link" /></p>

<p>Just like for registering passkeys, the page for authenticating via a passkey is already hooked up with the necessary JavaScript code, so clicking on the submit button should show a dialog for authenticating with the previously created passkey. Once I verify biometric identification, I’m authenticated with 2nd factor.</p>

<p><img src="/images/rodauth-passkey-authentication-dialog.png" alt="Rodauth passkey authentication dialog" /></p>

<h2 id="passwordless-login">Passwordless login</h2>

<p>Once the user has created a passkey for your website, in addition to multifactor authentication, they can also use it for passwordless login. This functionality is provided by the <a href="http://rodauth.jeremyevans.net/rdoc/files/doc/webauthn_login_rdoc.html"><code class="language-plaintext highlighter-rouge">webauthn_login</code></a> feature:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="n">enable</span> <span class="ss">:webauthn_login</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>This will automatically enable multi-phase login, where the user first enters their email, and then they can choose to authenticate via a password or a passkey. Note that the password field won’t be displayed if the user didn’t set one when creating their account.</p>

<p><img src="/images/rodauth-passkey-login.png" alt="Rodauth passkey login" /></p>

<p>Verifying the passkey will log the user in. If they’re using multifactor authentication, by default this will only authenticate 1st factor, so they’ll still need 2nd factor (for pages that require it). However, passkeys are generally considered multi-factor authentication, because the user presents something they “have” (device) and – if user verification took place – something they “are” (biometrics) or “know” (PIN).</p>

<p>We can tell Rodauth to consider user verification as 2nd factor. If there was no user verification, e.g. a security key was used that only requires user presence but doesn’t have biometrics or PIN, then only 1st factor will get authenticated.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="n">webauthn_login_user_verification_additional_factor?</span> <span class="kp">true</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>To make the login UX even better, WebAuthn protocol supports <a href="https://passkeys.dev/docs/reference/terms/#autofill-ui">autofill UI</a> for passkeys when the email field is focused. Rodauth once again has this built in via the <a href="http://rodauth.jeremyevans.net/rdoc/files/doc/webauthn_autofill_rdoc.html"><code class="language-plaintext highlighter-rouge">webauthn_autofill</code></a> feature (which happens to be a feature I added :blush:):</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="n">enable</span> <span class="ss">:webauthn_autofill</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>When the user opens the login page again and focuses the email field, they should see their passkey being offered. This is because stored passkeys include the user’s email address for identification.</p>

<figure>
  <img alt="Autofill UI on email field showing a dropdown with passkeyss" src="/images/rodauth-passkey-autofill.png" />
  <figcaption>There is normally a drop shadow, but my Mac's screen capture removed it.</figcaption>
</figure>

<p>When the user selects their passkey and verifies it, they’ll be automatically logged in, without even having to type their email address. In fact, they don’t even have to select the passkey, they can just scan their fingerprint as soon as they focus the email field.</p>

<p>In addition to passwordless login, Rodauth also supports passwordless registration via the <a href="http://rodauth.jeremyevans.net/rdoc/files/doc/webauthn_verify_account_rdoc.html"><code class="language-plaintext highlighter-rouge">webauthn_verify_account</code></a> feature. The user just needs to enter their email address in the create account form, and when they follow the link in the verification email, they’re required to register a passkey in order to verify their account.</p>

<h2 id="multiple-credentials">Multiple credentials</h2>

<p>Rodauth supports registering multiple passkeys for a single account; the user can just visit the WebAuthn setup page again and create another credential. This is useful for people wanting to register multiple devices for backup.</p>

<p>By default, credentials can only be differentiated by their last used timestamp, which is what’s displayed on the WebAuthn remove page by default.</p>

<p><img src="/images/rodauth-webauthn-remove.png" alt="Rodauth WebAuthn remove page" /></p>

<p>Let’s add the ability for the user to choose a nickname for their credentials, so that they can more easily differentiate between them. We’ll start by adding a new <code class="language-plaintext highlighter-rouge">nickname</code> column to the <code class="language-plaintext highlighter-rouge">account_webauthn_keys</code> table:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate migration add_nickname_to_account_webauthn_keys nickname:string
<span class="nv">$ </span>rails db:migrate
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">AddNicknameToAccountWebauthnKeys</span> <span class="o">&lt;</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Migration</span>
  <span class="k">def</span> <span class="nf">change</span>
    <span class="n">add_column</span> <span class="ss">:account_webauthn_keys</span><span class="p">,</span> <span class="ss">:nickname</span><span class="p">,</span> <span class="ss">:string</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Next, we’ll import the view templates used by the WebAuthn feature, and add a <code class="language-plaintext highlighter-rouge">nickname</code> field to the setup form:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate rodauth:views webauthn
<span class="c"># create  app/views/rodauth/webauthn_auth.html.erb</span>
<span class="c"># create  app/views/rodauth/webauthn_setup.html.erb</span>
<span class="c"># create  app/views/rodauth/webauthn_remove.html.erb</span>
</code></pre></div></div>
<div class="language-erb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- app/views/rodauth/webauthn_setup.html.erb --&gt;</span>
<span class="c">&lt;!-- ... --&gt;</span>
  <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"form-group mb-3"</span><span class="nt">&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">form</span><span class="p">.</span><span class="nf">label</span> <span class="ss">:nickname</span><span class="p">,</span> <span class="s2">"Nickname"</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"form-label"</span> <span class="cp">%&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">form</span><span class="p">.</span><span class="nf">text_field</span> <span class="ss">:nickname</span><span class="p">,</span> <span class="ss">value: </span><span class="n">params</span><span class="p">[</span><span class="ss">:nickname</span><span class="p">],</span> <span class="ss">class: </span><span class="s2">"form-control </span><span class="si">#{</span><span class="s2">"is-invalid"</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="s2">"nickname"</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span> <span class="ss">aria: </span><span class="p">({</span> <span class="ss">invalid: </span><span class="kp">true</span><span class="p">,</span> <span class="ss">describedby: </span><span class="s2">"nickname_error_message"</span> <span class="p">}</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="s2">"nickname"</span><span class="p">))</span> <span class="cp">%&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">content_tag</span><span class="p">(</span><span class="ss">:span</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="s2">"nickname"</span><span class="p">),</span> <span class="ss">class: </span><span class="s2">"invalid-feedback"</span><span class="p">,</span> <span class="ss">id: </span><span class="s2">"nickname_error_message"</span><span class="p">)</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="s2">"nickname"</span><span class="p">)</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;/div&gt;</span>
<span class="c">&lt;!-- ... --&gt;</span>
</code></pre></div></div>

<p><img src="/images/rodauth-passkey-nickname-field.png" alt="Rodauth passkey nickname field" /></p>

<p>Now we’ll hook into the WebAuthn setup request, validate that the <code class="language-plaintext highlighter-rouge">nickname</code> param was filled in and persist it on the new credential:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="n">before_webauthn_setup</span> <span class="k">do</span>
      <span class="n">throw_error_status</span><span class="p">(</span><span class="mi">422</span><span class="p">,</span> <span class="s2">"nickname"</span><span class="p">,</span> <span class="s2">"must be set"</span><span class="p">)</span> <span class="k">if</span> <span class="n">param</span><span class="p">(</span><span class="s2">"nickname"</span><span class="p">).</span><span class="nf">empty?</span>
    <span class="k">end</span>
    <span class="n">webauthn_key_insert_hash</span> <span class="k">do</span> <span class="o">|</span><span class="n">credential</span><span class="o">|</span>
      <span class="k">super</span><span class="p">(</span><span class="n">credential</span><span class="p">).</span><span class="nf">merge</span><span class="p">(</span><span class="ss">nickname: </span><span class="n">param</span><span class="p">(</span><span class="s2">"nickname"</span><span class="p">))</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Finally, let’s also modify the remove form to display nicknames instead of last used timestamps (we’re using the <code class="language-plaintext highlighter-rouge">Account#webauthn_keys</code> association defined by <a href="https://github.com/janko/rodauth-model">rodauth-model</a>):</p>

<div class="language-erb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- app/views/rodauth/webauthn_remove.html.erb --&gt;</span>
<span class="c">&lt;!-- ... --&gt;</span>
  <span class="nt">&lt;fieldset</span> <span class="na">class=</span><span class="s">"form-group mb-3"</span><span class="nt">&gt;</span>
    <span class="cp">&lt;%</span> <span class="n">current_account</span><span class="p">.</span><span class="nf">webauthn_keys</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">webauthn_key</span><span class="o">|</span> <span class="cp">%&gt;</span>
      <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"form-check"</span><span class="nt">&gt;</span>
        <span class="cp">&lt;%=</span> <span class="n">form</span><span class="p">.</span><span class="nf">radio_button</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">webauthn_remove_param</span><span class="p">,</span> <span class="n">webauthn_key</span><span class="p">.</span><span class="nf">webauthn_id</span><span class="p">,</span> <span class="ss">id: </span><span class="s2">"webauthn-remove-</span><span class="si">#{</span><span class="n">webauthn_key</span><span class="p">.</span><span class="nf">webauthn_id</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"form-check-input </span><span class="si">#{</span><span class="s2">"is-invalid"</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="n">rodauth</span><span class="p">.</span><span class="nf">webauthn_remove_param</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span> <span class="ss">aria: </span><span class="p">({</span> <span class="ss">invalid: </span><span class="kp">true</span><span class="p">,</span> <span class="ss">describedby: </span><span class="s2">"webauthn_remove_error_message"</span> <span class="p">}</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="n">rodauth</span><span class="p">.</span><span class="nf">webauthn_remove_param</span><span class="p">))</span> <span class="cp">%&gt;</span>
        <span class="cp">&lt;%=</span> <span class="n">form</span><span class="p">.</span><span class="nf">label</span> <span class="s2">"webauthn-remove-</span><span class="si">#{</span><span class="n">webauthn_key</span><span class="p">.</span><span class="nf">webauthn_id</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span> <span class="n">webauthn_key</span><span class="p">.</span><span class="nf">nickname</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"form-check-label"</span> <span class="cp">%&gt;</span>
        <span class="cp">&lt;%=</span> <span class="n">content_tag</span><span class="p">(</span><span class="ss">:span</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="n">rodauth</span><span class="p">.</span><span class="nf">webauthn_remove_param</span><span class="p">),</span> <span class="ss">class: </span><span class="s2">"invalid-feedback"</span><span class="p">,</span> <span class="ss">id: </span><span class="s2">"webauthn_remove_error_message"</span><span class="p">)</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="n">rodauth</span><span class="p">.</span><span class="nf">webauthn_remove_param</span><span class="p">)</span> <span class="o">&amp;&amp;</span> <span class="n">webauthn_key</span> <span class="o">==</span> <span class="n">current_account</span><span class="p">.</span><span class="nf">webauthn_keys</span><span class="p">.</span><span class="nf">last</span> <span class="cp">%&gt;</span>
      <span class="nt">&lt;/div&gt;</span>
    <span class="cp">&lt;%</span> <span class="k">end</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;/fieldset&gt;</span>
<span class="c">&lt;!-- ... --&gt;</span>
</code></pre></div></div>

<p><img src="/images/rodauth-passkey-remove-nicknames.png" alt="Rodauth passkey remove nicknames" /></p>

<h2 id="closing-words">Closing words</h2>

<p>Passkeys still need wider support in browsers and operating systems before they can become mainstream, but they look very promising. I like that I can use devices I already have, as opposed to having to buy a separate piece of hardware such as a YubiKey. I also feel safer that passkeys are synced automatically, and it’s convenient that I don’t have to remember on which Apple device I created a passkey for a given website.</p>

<p>The fact that Rodauth provides such advanced support for passkeys with zero configuration shows that it’s really keeping up with authentication trends. It also speaks to its incredibly flexible design, as passkeys can be combined with existing authentication methods, acting both as 1st and 2nd factor. The whole flow is highly configurable, as can be seen from the vast list of <a href="http://rodauth.jeremyevans.net/rdoc/files/doc/webauthn_rdoc.html">configuration methods</a>.</p>

<h2 id="resources">Resources</h2>

<ul>
  <li><a href="https://webauthn.io/">WebAuthn.io</a></li>
  <li><a href="https://fidoalliance.org/passkeys/">FIDO Alliance documentation</a></li>
  <li><a href="https://passkeys.dev/">Passkeys.dev</a></li>
  <li><a href="https://css-tricks.com/passkeys-what-the-heck-and-why/">Passkeys: What the Heck and Why?</a> (CSS Tricks)</li>
  <li><a href="https://blog.passwordless.id/webauthn-vs-passkeys">WebAuthn vs Passkeys</a> (Passwordless.ID)</li>
</ul>]]></content><author><name>Janko Marohnić</name><email>janko@hey.com</email></author><summary type="html"><![CDATA[Passkeys are a modern alternative to passwords, where the user’s device performs the authentication, usually requiring some form of user verification (biometric identification, PIN). Passkeys are built on top of WebAuthn specification, which is based on public-key cryptography. Keypairs are created for each website, and the public key is sent to the server, while the private key is securely stored on the device. This makes passkeys:]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://janko.io/images/social.jpg" /><media:content medium="image" url="https://janko.io/images/social.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Upgrading from Selenium to Cuprite</title><link href="https://janko.io/upgrading-from-selenium-to-cuprite/" rel="alternate" type="text/html" title="Upgrading from Selenium to Cuprite" /><published>2023-06-04T00:00:00+00:00</published><updated>2023-06-04T00:00:00+00:00</updated><id>https://janko.io/upgrading-from-selenium-to-cuprite</id><content type="html" xml:base="https://janko.io/upgrading-from-selenium-to-cuprite/"><![CDATA[<p>When I joined my current company, the system tests for our Rails app used Selenium as the Capybara driver. I didn’t have good experiences with Selenium in the past, mostly it was tedious to have to keep chromedriver up-to-date with the auto-updating Chrome. In this project, I was frequently hitting maximum number of open file descriptors on my OS when running system tests, probably in combination with Spring. We’re using the <a href="https://github.com/titusfortner/webdrivers">Webdrivers</a> gem, and we also needed to ignore its download URLs in VCR and WebMock. But my primary issue was that the system tests just seemed kind of slow in general.</p>

<p>I stumbled across <a href="https://www.bikeshed.fm/355">an episode of The Bike Shed podcast</a>, where it was mentioned that Selenium can add considerable overhead, so I decided it was worth trying out <a href="https://github.com/rubycdp/cuprite">Cuprite</a>. For those not familiar, Cuprite is a Capybara driver that interacts with Chrome directly using <a href="https://chromedevtools.github.io/devtools-protocol/">CDP</a> (Chrome DevTools Protocol). This is in contrast to Selenium, which goes through chromedriver/geckodriver command-line tool.</p>

<p>I have to say I did not expect the performance improvements to be so significant. Running individual system tests was about 30-50% faster on my M1 MacBook Air, while our overall system test suite went <strong>from 9 minutes to 6 minutes</strong>, which is a <strong>30% speedup</strong>. Just from changing the Capybara driver. For someone who cares about fast tests, this was a considerable win :metal:</p>

<p>Our Capybara configuration ended up being the following:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'capybara/rspec'</span>
<span class="nb">require</span> <span class="s1">'capybara/cuprite'</span>

<span class="no">Capybara</span><span class="p">.</span><span class="nf">default_max_wait_time</span> <span class="o">=</span> <span class="mi">5</span>
<span class="no">Capybara</span><span class="p">.</span><span class="nf">disable_animation</span> <span class="o">=</span> <span class="kp">true</span>

<span class="no">RSpec</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">config</span><span class="o">|</span>
  <span class="n">config</span><span class="p">.</span><span class="nf">before</span><span class="p">(</span><span class="ss">:each</span><span class="p">,</span> <span class="ss">type: :system</span><span class="p">)</span> <span class="k">do</span>
    <span class="n">driven_by</span><span class="p">(</span><span class="ss">:cuprite</span><span class="p">,</span> <span class="ss">screen_size: </span><span class="p">[</span><span class="mi">1440</span><span class="p">,</span> <span class="mi">810</span><span class="p">],</span> <span class="ss">options: </span><span class="p">{</span>
      <span class="ss">js_errors: </span><span class="kp">true</span><span class="p">,</span>
      <span class="ss">headless: </span><span class="sx">%w[0 false]</span><span class="p">.</span><span class="nf">exclude?</span><span class="p">(</span><span class="no">ENV</span><span class="p">[</span><span class="s2">"HEADLESS"</span><span class="p">]),</span>
      <span class="ss">slowmo: </span><span class="no">ENV</span><span class="p">[</span><span class="s2">"SLOWMO"</span><span class="p">]</span><span class="o">&amp;</span><span class="p">.</span><span class="nf">to_f</span><span class="p">,</span>
      <span class="ss">process_timeout: </span><span class="mi">15</span><span class="p">,</span>
      <span class="ss">timeout: </span><span class="mi">10</span><span class="p">,</span>
      <span class="ss">browser_options: </span><span class="no">ENV</span><span class="p">[</span><span class="s2">"DOCKER"</span><span class="p">]</span> <span class="p">?</span> <span class="p">{</span> <span class="s2">"no-sandbox"</span> <span class="o">=&gt;</span> <span class="kp">nil</span> <span class="p">}</span> <span class="p">:</span> <span class="p">{}</span>
    <span class="p">})</span>
  <span class="k">end</span>

  <span class="n">config</span><span class="p">.</span><span class="nf">filter_gems_from_backtrace</span><span class="p">(</span><span class="s2">"capybara"</span><span class="p">,</span> <span class="s2">"cuprite"</span><span class="p">,</span> <span class="s2">"ferrum"</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>

<h2 id="flaky-test-failures">Flaky test failures</h2>

<p>While moving to Cuprite made tests faster, we started getting dozens of new flaky test failures. After looking at them more closely, I found that each failure was caused by improper waiting for JavaScript (we’re using Hotwire). With Selenium, those issues were just masked, because the overhead was big enough that the race conditions never manifested.</p>

<p>Most failures were around clicking on a link and then waiting for text to appear, but that text was already present on the previous page, so Capybara didn’t actually wait for the new page to get navigated to. In some cases we needed to manually scroll to elements before interacting with them, where Selenium appears to have automatically scrolled.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">click_on</span> <span class="s2">"Some link"</span>
<span class="c1"># make sure this text was NOT present on the previous page</span>
<span class="n">expect</span><span class="p">(</span><span class="n">page</span><span class="p">).</span><span class="nf">to</span> <span class="n">have_content</span><span class="p">(</span><span class="s2">"Some text"</span><span class="p">)</span>
</code></pre></div></div>

<h3 id="disabling-css-transitions">Disabling CSS transitions</h3>

<p>We’re using Bootstrap, and I noticed that some modal clicks were failing. It turned out that due to CSS transitions used by Bootstrap modals, Capybara would sometimes attempt to click on a moving target. Since we don’t actually need animations in tests, I was looking for a way to disable them. By sheer luck I discovered a handy Capybara configuration that takes care of this:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># disable CSS transitions and jQuery animations</span>
<span class="no">Capybara</span><span class="p">.</span><span class="nf">disable_animation</span> <span class="o">=</span> <span class="kp">true</span>
</code></pre></div></div>

<p>One problem is that our flash messages are set to fade away after 3 seconds, so this caused them not to disappear automatically. This was fine most of the time, but in some cases they were covering content we needed to interact with. To address this, I created a helper method that retrieves the flash message and immediatelly closes the alert:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">flash_message</span>
  <span class="n">message</span> <span class="o">=</span> <span class="n">find</span><span class="p">(</span><span class="s2">".flash"</span><span class="p">).</span><span class="nf">text</span><span class="p">.</span><span class="nf">split</span><span class="p">(</span><span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">).</span><span class="nf">last</span>
  <span class="n">find</span><span class="p">(</span><span class="s2">".flash .close"</span><span class="p">).</span><span class="nf">click</span> <span class="c1"># close alert</span>
  <span class="n">message</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># ...</span>
<span class="n">click_on</span> <span class="s2">"Create Device"</span>
<span class="n">expect</span><span class="p">(</span><span class="n">flash_message</span><span class="p">).</span><span class="nf">to</span> <span class="n">eq</span> <span class="s2">"Device was successfully created"</span>
</code></pre></div></div>

<h3 id="disabling-turbo-previews">Disabling Turbo previews</h3>

<p>In a previous project, I found that <a href="https://turbo.hotwired.dev/handbook/building#opting-out-of-caching">Turbo previews</a> were the cause of one flaky test, so I disabled them by adding the following into <code class="language-plaintext highlighter-rouge">&lt;head&gt;</code> of the layout:</p>

<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- disable cached Turbo previews when navigating --&gt;</span>
<span class="nt">&lt;meta</span> <span class="na">name=</span><span class="s">"turbo-cache-control"</span> <span class="na">content=</span><span class="s">"no-preview"</span><span class="nt">&gt;</span>
</code></pre></div></div>

<h2 id="raising-stimulus-errors">Raising Stimulus errors</h2>

<p>When a JavaScript errors occur inside Stimulus lifecycle callbacks or actions, they are <a href="https://stimulus.hotwired.dev/handbook/installing#error-handling">caught</a> by Stimulus and logged. This avoids an error from one Stimulus controller halting JavaScript execution and preventing other Stimulus controllers from being executed (see <a href="https://github.com/hotwired/stimulus/issues/236#issuecomment-479694545">Sam Stephenson’s explanation</a> for more details).</p>

<p>While this is useful for production, in tests we want to get alerted when JavaScript errors occur. After configuring Cuprite to convert any JS errors into Ruby exceptions by setting <code class="language-plaintext highlighter-rouge">js_errors: true</code>, we overrode Stimulus application’s error handler in tests to propagate errors:</p>

<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">import</span> <span class="p">{</span> <span class="nx">Application</span> <span class="p">}</span> <span class="k">from</span> <span class="dl">"</span><span class="s2">@hotwired/stimulus</span><span class="dl">"</span>

<span class="kd">const</span> <span class="nx">application</span> <span class="o">=</span> <span class="nx">Application</span><span class="p">.</span><span class="nf">start</span><span class="p">()</span>

<span class="c1">// Works with Webpacker (in Vite we'd use `import.meta.env.MODE === "test"`).</span>
<span class="k">if </span><span class="p">(</span><span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">RAILS_ENV</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">test</span><span class="dl">"</span><span class="p">)</span> <span class="p">{</span>
  <span class="c1">// propagate errors that happen inside Stimulus controllers</span>
  <span class="nx">application</span><span class="p">.</span><span class="nx">handleError</span> <span class="o">=</span> <span class="p">(</span><span class="nx">error</span><span class="p">,</span> <span class="nx">message</span><span class="p">,</span> <span class="nx">detail</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
    <span class="k">throw</span> <span class="nx">error</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<h2 id="precompiling-assets">Precompiling assets</h2>

<p>We were initially hitting timeout errors on CI from Cuprite on the first test. The reason was that Webpacker would compile assets on the first request. We tried increasing <code class="language-plaintext highlighter-rouge">:process_timeout</code> in Cuprite, but that didn’t help.</p>

<p>Instead of letting Webpacker compile assets on-the-fly, we chose to precompile them in advance, which fixed the issue.</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>bundle <span class="nb">exec </span>rake assets:precompile
<span class="nv">$ </span>bundle <span class="nb">exec </span>rspec spec/system
</code></pre></div></div>

<p>However, when JS errors would occur, I noticed the JavaScript source was now minified. This not only made it more difficult to locate where the error occurred, but on CI the error message was completely absent. This is because the <code class="language-plaintext highlighter-rouge">webpack:compile</code> rake task <a href="https://github.com/rails/webpacker/blob/3fd96bcbf495db5a24a46606465e9837fec232c1/lib/tasks/webpacker/compile.rake#L23">defaults</a> <code class="language-plaintext highlighter-rouge">NODE_ENV</code> to <code class="language-plaintext highlighter-rouge">production</code>. First I tried setting <code class="language-plaintext highlighter-rouge">NODE_ENV=test</code>, but that didn’t skip minification, and I later found out this is intended for JavaScript tests. Setting <code class="language-plaintext highlighter-rouge">NODE_ENV=development</code> on CI worked, which is what’s used for on-the-fly compilation.</p>

<p>I would prefer Webpacker not to merge assets into a single file in tests, but I think migrating to <a href="https://vite-ruby.netlify.app/">Vite</a> would help with this.</p>

<h2 id="toggling-headless-mode">Toggling headless mode</h2>

<p>Cuprite runs Chrome in so-called “headless” mode by default, which means the browser doesn’t open up while tests are being run. However, if you’re debugging a failing test, it’s not always enough to look at the captured screenshots, sometimes you need to <em>see</em> what’s happening on the page.</p>

<p>Our Cuprite configuration allowed us to pass <code class="language-plaintext highlighter-rouge">HEADLESS=0</code> environment variable to the <code class="language-plaintext highlighter-rouge">rspec</code> command to disable headless mode. If the interaction was happening too fast to make sense of anything, we could additionally set e.g. <code class="language-plaintext highlighter-rouge">SLOWMO=0.5</code> to add 0.5s of overhead to every click.</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ HEADLESS</span><span class="o">=</span>0 <span class="nv">SLOWMO</span><span class="o">=</span>0.5 bin/rspec spec/system/something_spec.rb
</code></pre></div></div>

<h2 id="docker-handling">Docker handling</h2>

<p>Some team members prefer to run the Rails app locally through Docker. To make Cuprite work, we set <code class="language-plaintext highlighter-rouge">DOCKER=true</code> in our <code class="language-plaintext highlighter-rouge">docker-compose.yml</code>, and then based on that environment variable passed the <code class="language-plaintext highlighter-rouge">no-sandbox</code> option to Chrome.</p>

<h2 id="closing-words">Closing words</h2>

<p>The performance benefits alone will definitely make me advocate for using Cuprite every time. The only feature I found it doesn’t support yet is drag-and-drop, though <a href="https://github.com/rubycdp/cuprite/pull/176">initial support</a> has been merged to master.</p>

<p>Flaky test failures have always been a challenge for me when writing system tests. What helped me is to trust that the root cause is always figureoutable. I would previously attribute them to complex Selenium internals, but Cuprite is much less magical in comparison, so I can make better sense of why something is happening.</p>]]></content><author><name>Janko Marohnić</name><email>janko@hey.com</email></author><summary type="html"><![CDATA[When I joined my current company, the system tests for our Rails app used Selenium as the Capybara driver. I didn’t have good experiences with Selenium in the past, mostly it was tedious to have to keep chromedriver up-to-date with the auto-updating Chrome. In this project, I was frequently hitting maximum number of open file descriptors on my OS when running system tests, probably in combination with Spring. We’re using the Webdrivers gem, and we also needed to ignore its download URLs in VCR and WebMock. But my primary issue was that the system tests just seemed kind of slow in general.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://janko.io/images/social.jpg" /><media:content medium="image" url="https://janko.io/images/social.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Social Login in Rails with Rodauth</title><link href="https://janko.io/social-login-in-rails-with-rodauth/" rel="alternate" type="text/html" title="Social Login in Rails with Rodauth" /><published>2022-12-06T00:00:00+00:00</published><updated>2022-12-06T00:00:00+00:00</updated><id>https://janko.io/social-login-in-rails-with-rodauth</id><content type="html" xml:base="https://janko.io/social-login-in-rails-with-rodauth/"><![CDATA[<p><a href="https://github.com/omniauth/omniauth">OmniAuth</a> provides a standardized interface for authenticating with various external providers. Once the user authenticates with the provider, it’s up to us developers to handle the callback and implement actual login and registration into the app. There is a <a href="https://github.com/omniauth/omniauth/wiki/Managing-Multiple-Providers">wiki page</a> laying out various scenarios that need to be handled if you want to support multiple providers, showing that it’s by no means a trivial task.</p>

<p>While Devise provides a convenience layer around OmniAuth, it does nothing to actually sign the user into your app. When I started writing the OmniAuth integration for <a href="https://github.com/jeremyevans/rodauth">Rodauth</a>, I wanted to go one step further and actually handle things like persistence of external identities, account creation and login, while still allowing the developer to customize the behaviour. That’s how <a href="https://github.com/janko/rodauth-omniauth">rodauth-omniauth</a> was created. :sparkles:</p>

<p>In this article, I will show how to add social login to an existing Rails app that’s using Rodauth, and extend the default behaviour with some custom logic. If you’re looking to get started with Rodauth, check out <a href="https://janko.io/adding-authentication-in-rails-with-rodauth/">my previous article</a>. With that out of the way, let’s dive in.</p>

<h2 id="setup">Setup</h2>

<p>We’ll start by installing the Rodauth extension and the desired OmniAuth strategies:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>bundle add rodauth-omniauth omniauth-facebook omniauth-google-oauth2
</code></pre></div></div>

<p>You don’t need to install any gems for CSRF protection of OmniAuth request endpoints, because rodauth-omniauth will automatically use whichever CSRF protection mechanism Rodauth was configured with, which in case of Rails will be <code class="language-plaintext highlighter-rouge">ActionController::RequestForgeryProtection</code>.</p>

<p>If you haven’t already, create the necessary OAuth apps, and configure the callback URL to be <code class="language-plaintext highlighter-rouge">https://localhost:3000/auth/{provider}/callback</code>, where <code class="language-plaintext highlighter-rouge">{provider}</code> is either <code class="language-plaintext highlighter-rouge">facebook</code> or <code class="language-plaintext highlighter-rouge">google</code>. We’ll save the credentials for the OAuth apps into our project:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails credentials:edit
</code></pre></div></div>
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># ...</span>
<span class="na">facebook</span><span class="pi">:</span>
  <span class="na">app_id</span><span class="pi">:</span> <span class="s2">"</span><span class="s">&lt;YOUR_APP_ID&gt;"</span>
  <span class="na">app_secret</span><span class="pi">:</span> <span class="s2">"</span><span class="s">&lt;YOUR_APP_SECRET&gt;"</span>
<span class="na">google</span><span class="pi">:</span>
  <span class="na">client_id</span><span class="pi">:</span> <span class="s2">"</span><span class="s">&lt;YOUR_CLIENT_ID&gt;"</span>
  <span class="na">client_secret</span><span class="pi">:</span> <span class="s2">"</span><span class="s">&lt;YOUR_CLIENT_SECRET&gt;"</span>
</code></pre></div></div>

<p>Next, we’ll need to create the table that rodauth-omniauth will use for storing external identities:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate migration create_account_identities
<span class="nv">$ </span>rails db:migrate
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">CreateAccountIdentities</span> <span class="o">&lt;</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Migration</span>
  <span class="k">def</span> <span class="nf">change</span>
    <span class="n">create_table</span> <span class="ss">:account_identities</span> <span class="k">do</span> <span class="o">|</span><span class="n">t</span><span class="o">|</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">references</span> <span class="ss">:account</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span><span class="p">,</span> <span class="ss">foreign_key: </span><span class="p">{</span> <span class="ss">on_delete: :cascade</span> <span class="p">}</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">string</span> <span class="ss">:provider</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">string</span> <span class="ss">:uid</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">index</span> <span class="p">[</span><span class="ss">:provider</span><span class="p">,</span> <span class="ss">:uid</span><span class="p">],</span> <span class="ss">unique: </span><span class="kp">true</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>In the Rodauth configuration, we can now enable the <code class="language-plaintext highlighter-rouge">omniauth</code> feature and register our strategies:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="n">enable</span> <span class="ss">:omniauth</span>

    <span class="n">omniauth_provider</span> <span class="ss">:facebook</span><span class="p">,</span>
      <span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">credentials</span><span class="p">.</span><span class="nf">facebook</span><span class="p">[</span><span class="ss">:app_id</span><span class="p">],</span>
      <span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">credentials</span><span class="p">.</span><span class="nf">facebook</span><span class="p">[</span><span class="ss">:app_secret</span><span class="p">],</span>
      <span class="ss">scope: </span><span class="s2">"email"</span>

    <span class="n">omniauth_provider</span> <span class="ss">:google_oauth2</span><span class="p">,</span>
      <span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">credentials</span><span class="p">.</span><span class="nf">google</span><span class="p">[</span><span class="ss">:client_id</span><span class="p">],</span>
      <span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">credentials</span><span class="p">.</span><span class="nf">google</span><span class="p">[</span><span class="ss">:client_secret</span><span class="p">],</span>
      <span class="ss">name: :google</span> <span class="c1"># rename it from "google_oauth2"</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Finally, we can import the view templates for the login form, and add the social login links there:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate rodauth:views login
<span class="c">#  create  app/views/rodauth/_login_form.html.erb</span>
<span class="c">#  create  app/views/rodauth/_login_form_footer.html.erb</span>
<span class="c">#  create  app/views/rodauth/_login_form_footer.html.erb</span>
<span class="c">#  create  app/views/rodauth/_login_form_header.html.erb</span>
<span class="c">#  create  app/views/rodauth/login.html.erb</span>
<span class="c">#  create  app/views/rodauth/multi_phase_login.html.erb</span>
</code></pre></div></div>
<div class="language-erb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- app/views/rodauth/_login_form_footer.html.erb --&gt;</span>
<span class="c">&lt;!-- ... --&gt;</span>
  <span class="nt">&lt;li&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">button_to</span> <span class="s2">"Login via Facebook"</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">omniauth_request_path</span><span class="p">(</span><span class="ss">:facebook</span><span class="p">),</span>
      <span class="ss">method: :post</span><span class="p">,</span> <span class="ss">data: </span><span class="p">{</span> <span class="ss">turbo: </span><span class="kp">false</span> <span class="p">},</span> <span class="ss">class: </span><span class="s2">"btn btn-link p-0"</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;/li&gt;</span>
  <span class="nt">&lt;li&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">button_to</span> <span class="s2">"Login via Google"</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">omniauth_request_path</span><span class="p">(</span><span class="ss">:google</span><span class="p">),</span>
      <span class="ss">method: :post</span><span class="p">,</span> <span class="ss">data: </span><span class="p">{</span> <span class="ss">turbo: </span><span class="kp">false</span> <span class="p">},</span> <span class="ss">class: </span><span class="s2">"btn btn-link p-0"</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;/li&gt;</span>
<span class="c">&lt;!-- ... --&gt;</span>
</code></pre></div></div>

<p>We’re using POST form submits, because OmniAuth doesn’t allow GET requests for the request phase anymore by default. You’ll notice we had to disable Turbo for the request links, as those redirect to an external authorize URL, which don’t support AJAX visits.</p>

<p>Some OAuth authorizations require that the web app is served over HTTPS. Assuming you’re using Puma, a quick way to enable this locally would be to install the <a href="https://github.com/socketry/localhost">localhost</a> gem, and tell the Rails server you want to use SSL:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>bundle add localhost <span class="nt">--group</span> development
<span class="nv">$ </span>rails server <span class="nt">-b</span> ssl://localhost:3000
</code></pre></div></div>

<p>Now you should be able to open <code class="language-plaintext highlighter-rouge">https://localhost:3000/login</code>, and see the Rodauth login page with social login links.</p>

<p><img src="/images/social-login-links.png" alt="Login page with Facebook and Google login links" /></p>

<h2 id="login--registration">Login &amp; Registration</h2>

<p>When we visit the Facebook login link, and authorize the OAuth app, upon returning to the app a new verified account with your Facebook email address should be automatically created, along with the external identity, and you should be logged in.</p>

<p>You should see something like this in the database:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">account</span> <span class="o">=</span> <span class="no">Account</span><span class="p">.</span><span class="nf">last</span>
<span class="c1">#=&gt; #&lt;Account id: 123, status: "verified", email: "janko.marohnic@gmail.com"&gt;</span>
<span class="n">account</span><span class="p">.</span><span class="nf">identities</span>
<span class="c1">#=&gt; [#&lt;Account::Identity id: 456, account_id: 123, provider: "facebook", uid: "350872771"&gt;]</span>
</code></pre></div></div>

<p>A problem I often experienced as a user was forgetting which social provider I initially logged into on a certain app, and whether I had even logged in with a social provider (though I can now easily determine the latter by checking my password manager). If I would sign in with the wrong provider, this would usually result in a new account being created for me.</p>

<p>Wouldn’t it be nice if the app could detect that the email address of my existing account matches the one of the external identity, and automatically assign that identity to the existing account? Well, rodauth-omniauth does that automatically. If I were to authenticate with Google the next time, I would get logged into my existing account, with the new Google identity connected to it.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">account</span><span class="p">.</span><span class="nf">identities</span> <span class="c1">#=&gt; [</span>
<span class="c1">#   #&lt;Account::Identity id: 456, account_id: 123, provider: "facebook", uid: "350872771"&gt;,</span>
<span class="c1">#   #&lt;Account::Identity id: 789, account_id: 123, provider: "google", uid: "987349876343"&gt;,</span>
<span class="c1"># ]</span>
</code></pre></div></div>

<p>If for whatever reason you want to change or disable this behaviour, you can override <code class="language-plaintext highlighter-rouge">account_from_omniauth</code>, which is what searches existing accounts when authenticating with a provider for the first time:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># ...</span>
    <span class="n">account_from_omniauth</span> <span class="k">do</span>
      <span class="c1"># this is roughly the default implementation</span>
      <span class="n">account_table_ds</span><span class="p">.</span><span class="nf">first</span><span class="p">(</span><span class="ss">email: </span><span class="n">omniauth_info</span><span class="p">[</span><span class="s2">"email"</span><span class="p">])</span>
    <span class="k">end</span>
    <span class="c1"># OR</span>
    <span class="n">account_from_omniauth</span> <span class="p">{}</span> <span class="c1"># new identity = new account</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<h2 id="storing-additional-data">Storing additional data</h2>

<p>Let’s say users in our app can fill in their full name, and we decided to inherit it from their external identity when possible, to save them extra work.</p>

<p>We’ll assume we have a separate <code class="language-plaintext highlighter-rouge">profiles</code> table associated to accounts, and we’re already creating a profile record on normal registration:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># in a migration:</span>
<span class="n">create_table</span> <span class="ss">:profiles</span> <span class="k">do</span> <span class="o">|</span><span class="n">t</span><span class="o">|</span>
  <span class="n">t</span><span class="p">.</span><span class="nf">references</span> <span class="ss">:account</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span><span class="p">,</span> <span class="ss">foreign_key: </span><span class="kp">true</span>
  <span class="n">t</span><span class="p">.</span><span class="nf">string</span> <span class="ss">:name</span>
  <span class="n">t</span><span class="p">.</span><span class="nf">timestamps</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># create profile after normal registration</span>
    <span class="n">after_create_account</span> <span class="p">{</span> <span class="no">Profile</span><span class="p">.</span><span class="nf">create!</span><span class="p">(</span><span class="ss">account_id: </span><span class="n">account_id</span><span class="p">)</span> <span class="p">}</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>We can override the hook for creating the account via OmniAuth login, and create the profile with the name prefilled:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># create profile after registration through OmniAuth login</span>
    <span class="n">after_omniauth_create_account</span> <span class="k">do</span>
      <span class="no">Profile</span><span class="p">.</span><span class="nf">create!</span><span class="p">(</span><span class="ss">account_id: </span><span class="n">account_id</span><span class="p">,</span> <span class="ss">name: </span><span class="n">omniauth_info</span><span class="p">[</span><span class="s2">"name"</span><span class="p">])</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Let’s say we now want to store additional data on identities, which by default have only the mandatory <code class="language-plaintext highlighter-rouge">provider</code> and <code class="language-plaintext highlighter-rouge">uid</code> columns. We might want to store <code class="language-plaintext highlighter-rouge">created_at</code> &amp; <code class="language-plaintext highlighter-rouge">updated_at</code> timestamps, as well as an <code class="language-plaintext highlighter-rouge">email</code> for each identity. We can start by creating the necessary columns:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># in a migration:</span>
<span class="n">add_timestamps</span> <span class="ss">:account_identities</span>
<span class="n">add_column</span> <span class="ss">:account_identities</span><span class="p">,</span> <span class="ss">:email</span><span class="p">,</span> <span class="ss">:string</span>
</code></pre></div></div>

<p>We can now ensure <code class="language-plaintext highlighter-rouge">created_at</code> is set the first time we authenticate with a provider, and that <code class="language-plaintext highlighter-rouge">updated_at</code> and <code class="language-plaintext highlighter-rouge">email</code> are updated each time we authenticate with the provider:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># store `created_at` only when the identity is created</span>
    <span class="n">omniauth_identity_insert_hash</span> <span class="k">do</span>
      <span class="k">super</span><span class="p">().</span><span class="nf">merge</span><span class="p">(</span><span class="ss">created_at: </span><span class="no">Time</span><span class="p">.</span><span class="nf">now</span><span class="p">)</span>
    <span class="k">end</span>
    <span class="c1"># update `updated_at` and `email` each time the identity is updated</span>
    <span class="n">omniauth_identity_update_hash</span> <span class="k">do</span>
      <span class="k">super</span><span class="p">().</span><span class="nf">merge</span><span class="p">(</span><span class="ss">updated_at: </span><span class="no">Time</span><span class="p">.</span><span class="nf">now</span><span class="p">,</span> <span class="ss">email: </span><span class="n">omniauth_email</span><span class="p">)</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Now our account &amp; identity data might look as follows:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">account</span> <span class="o">=</span> <span class="no">Account</span><span class="p">.</span><span class="nf">last</span>
<span class="c1">#=&gt; #&lt;Account id: 123,</span>
<span class="c1">#    status: "verified",</span>
<span class="c1">#    email: "janko@hey.com",</span>
<span class="c1">#    name: "Janko Marohnić"&gt;</span>

<span class="n">account</span><span class="p">.</span><span class="nf">identities</span><span class="p">.</span><span class="nf">first</span>
<span class="c1">#=&gt; #&lt;Account::Identity</span>
<span class="c1">#    id: 789,</span>
<span class="c1">#    account_id: 123,</span>
<span class="c1">#    provider: "google",</span>
<span class="c1">#    uid: "987349876343",</span>
<span class="c1">#    email: "janko.marohnic@gmail.com",</span>
<span class="c1">#    created_at: Fri, 11 Nov 2022 13:11:85 UTC,</span>
<span class="c1">#    updated_at: Fri, 02 Dec 2022 08:01:26 UTC&gt;</span>
</code></pre></div></div>

<h2 id="multiple-account-types">Multiple account types</h2>

<p>If your app has different account types which require potentially different authentication rules, you’ll be glad to know that rodauth-omniauth supports distinct configurations.</p>

<p>Let’s say we have an <strong>admin</strong> account type, for which we want to provide logging in via GitHub. Assuming we’ve already created the OAuth app, we’ll install the OmniAuth strategy gem:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>bundle add omniauth-github
<span class="nv">$ </span>rails credentials:edit
</code></pre></div></div>
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># ...</span>
<span class="na">github</span><span class="pi">:</span>
  <span class="na">client_id</span><span class="pi">:</span> <span class="s2">"</span><span class="s">&lt;YOUR_CLIENT_ID&gt;"</span>
  <span class="na">client_secret</span><span class="pi">:</span> <span class="s2">"</span><span class="s">&lt;YOUR_CLIENT_SECRET&gt;"</span>
</code></pre></div></div>

<p>To protect the admin section, we’ll only allow users that are members of the company’s GitHub organization. For this, we might have the following Rodauth configuration:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_admin.rb</span>
<span class="k">class</span> <span class="nc">RodauthAdmin</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="n">enable</span> <span class="ss">:omniauth</span>

    <span class="n">prefix</span> <span class="s2">"/admin"</span>
    <span class="n">session_key_prefix</span> <span class="s2">"admin_"</span>

    <span class="n">omniauth_provider</span> <span class="ss">:github</span><span class="p">,</span>
      <span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">credentials</span><span class="p">.</span><span class="nf">github</span><span class="p">[</span><span class="ss">:client_id</span><span class="p">],</span>
      <span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">credentials</span><span class="p">.</span><span class="nf">github</span><span class="p">[</span><span class="ss">:client_secret</span><span class="p">]</span>

    <span class="n">before_omniauth_callback_route</span> <span class="k">do</span>
      <span class="k">if</span> <span class="n">omniauth_provider</span> <span class="o">==</span> <span class="ss">:github</span> <span class="o">&amp;&amp;</span> <span class="o">!</span><span class="n">organization_member?</span><span class="p">(</span><span class="n">omniauth_info</span><span class="p">[</span><span class="s2">"nickname"</span><span class="p">])</span>
        <span class="n">set_redirect_error_flash</span> <span class="s2">"User is not a member of our GitHub organization"</span>
        <span class="n">redirect</span> <span class="s2">"/admin"</span>
      <span class="k">end</span>
    <span class="k">end</span>
  <span class="k">end</span>

  <span class="kp">private</span>

  <span class="k">def</span> <span class="nf">organization_member?</span><span class="p">(</span><span class="n">username</span><span class="p">)</span>
    <span class="c1"># ... check if user is a member of company's GitHub organization ...</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Since we’ve set admin routes to be prefixed with <code class="language-plaintext highlighter-rouge">/admin</code>, OmniAuth routes will be prefixed as well, so the request phase will be at <code class="language-plaintext highlighter-rouge">/admin/auth/github</code>. This ensures authentication doesn’t overlap with the main account type.</p>

<p>If we had decided to use a separate table for admin accounts (e.g. <code class="language-plaintext highlighter-rouge">admins</code>), we can also use a separate identities table:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># in a migration:</span>
<span class="n">create_table</span> <span class="ss">:admin_identities</span> <span class="k">do</span> <span class="o">|</span><span class="n">t</span><span class="o">|</span>
  <span class="n">t</span><span class="p">.</span><span class="nf">references</span> <span class="ss">:admin</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span><span class="p">,</span> <span class="ss">foreign_key: </span><span class="p">{</span> <span class="ss">on_delete: :cascade</span> <span class="p">}</span>
  <span class="n">t</span><span class="p">.</span><span class="nf">string</span> <span class="ss">:provider</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span>
  <span class="n">t</span><span class="p">.</span><span class="nf">string</span> <span class="ss">:uid</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span>
  <span class="n">t</span><span class="p">.</span><span class="nf">index</span> <span class="p">[</span><span class="ss">:provider</span><span class="p">,</span> <span class="ss">:uid</span><span class="p">],</span> <span class="ss">unique: </span><span class="kp">true</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">RodauthAdmin</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># ...</span>
    <span class="n">omniauth_identities_table</span> <span class="ss">:admin_identities</span>
    <span class="n">omniauth_identities_account_id_column</span> <span class="ss">:admin_id</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<h2 id="future-work">Future work</h2>

<h3 id="separate-registration-step">Separate registration step</h3>

<p>The current automatic registration assumes the external login will always return the user’s email address, which is not always the case (hello Twitter). It also assumes we don’t need additional information from the user before creating their account.</p>

<p>After external login, I would like to support having a separate registration step, with fields like email address already being filled in. The main challenge is preventing an attacker from signing up with another person’s identity. I would definitely welcome any contributions. :pray:</p>

<h3 id="connecting-additional-identities">Connecting additional identities</h3>

<p>Once the user is signed in, it would be useful to allow them to connect additional external identities, as well as disconnect already linked identities, to make future logins more reliable.</p>

<figure>
  <img alt="GitLab section for connection &amp; disconnecting additional external identities" src="/images/connecting-identities.png" />
  <figcaption>GitLab account interface</figcaption>
</figure>

<p>This feature seems more straightforward to implement. However, it’s tricky to handle the scenario when a logged in user wants to authenticate via a different provider, because by default Rodauth doesn’t disallow access to the login page in this case. There are also questions around connecting identities that are currently assigned to a different account.</p>

<h2 id="closing-words">Closing words</h2>

<p>This was definitely the most challenging Rodauth extension I’ve built. I had been working on it periodically for 2 years, and was only able to release it once I decided to postpone the mentioned features. It took a while to find the right balance between respecting OmniAuth configuration and Rodauth conventions, support JSON API with JWT, handle inheritance, figure out the OmniAuth 2.0 upgrade, and implement a customizable callback phase.</p>

<p>I was able to do all this thanks to the strong foundation Rodauth provides. Its layered design allowed me to hook at the right level to make it work well with other authentication features, while the <a href="http://rodauth.jeremyevans.net/rdoc/files/doc/guides/internals_rdoc.html#label-Feature+Creation+Example">configuration DSL</a> made it easy to make any part customizable. Because Rodauth supports feature dependencies, I was able to extract the <a href="https://github.com/janko/rodauth-omniauth#base">pure OmniAuth integration</a> for standalone usage, for those who don’t need any of the database logic.</p>

<p>While it was previously possible to use OmniAuth directly, I’m happy that Rodauth now has a social login story. Given that this is a much more integrated solution compared to what Devise offers, and other people might have more custom authentication flows, I’m curious to get everyone’s feedback. :wink:</p>]]></content><author><name>Janko Marohnić</name><email>janko@hey.com</email></author><category term="rodauth" /><summary type="html"><![CDATA[OmniAuth provides a standardized interface for authenticating with various external providers. Once the user authenticates with the provider, it’s up to us developers to handle the callback and implement actual login and registration into the app. There is a wiki page laying out various scenarios that need to be handled if you want to support multiple providers, showing that it’s by no means a trivial task.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://janko.io/images/social.jpg" /><media:content medium="image" url="https://janko.io/images/social.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">What It Took to Build a Rails Integration for Rodauth</title><link href="https://janko.io/what-it-took-to-build-a-rails-integration-for-rodauth/" rel="alternate" type="text/html" title="What It Took to Build a Rails Integration for Rodauth" /><published>2022-10-12T00:00:00+00:00</published><updated>2022-10-12T00:00:00+00:00</updated><id>https://janko.io/what-it-took-to-build-a-rails-integration-for-rodauth</id><content type="html" xml:base="https://janko.io/what-it-took-to-build-a-rails-integration-for-rodauth/"><![CDATA[<p>When <a href="https://github.com/jeremyevans/rodauth">Rodauth</a> came out, I was excited to finally have a full-featured authentication framework that <em>wasn’t</em> tied to Rails, given that existing solutions required either Rails (Devise, Sorcery), or at least Active Record (Authlogic). Even though I mainly develop in Rails, I want other Ruby web frameworks to be viable alternatives, so I’m naturally drawn to generic solutions that everyone can use.</p>

<p>Even though Rodauth is built on top of <a href="https://github.com/jeremyevans/roda">Roda</a> and <a href="https://github.com/jeremyevans/sequel">Sequel</a>, it can work as a Rack middleware in any Ruby web framework. In the beginning, there was a <a href="https://github.com/jeremyevans/rodauth-demo-rails/blob/master/config/initializers/rodauth.rb">demo app</a> showing how Rodauth can be used in Rails, which leveraged the (now discontinued) <a href="https://github.com/jeremyevans/roda-rails">roda-rails</a> gem. However, the integration felt fairly raw, and definitely lacked the ergonomics Rails developers are used to.</p>

<p>Rodauth has a <a href="https://rodauth.jeremyevans.net/documentation.html#plugins">vast feature set</a>, but if it was going to compete with other authentication solutions, it needed to match their level of convenience in context of Rails. That meant deeply integrating into the Rails framework, having clear drawers where code goes, and defaults that are easy to get started with. At the start of 2020, I set on a mission to make using Rodauth in Rails easy.</p>

<h2 id="initial-spike">Initial spike</h2>

<p>I first created a <a href="https://github.com/janko/rodauth-demo-rails">demo Rails app</a>, and started setting up Rodauth there. In the <a href="https://github.com/janko/rodauth-demo-rails/commit/8affb9499801b6d2545f57b1554295f03502d2fa#diff-3c9dfcead9f75d230dc142b713a4bfc1e95faf07a3a027176b46519d95468453">early</a> <a href="https://github.com/janko/rodauth-demo-rails/commit/e69f9bd2149760d4d5e47ecf23d6239dc5448497#diff-c2af93c5a8391da5143ed228699395dc7ac8722b9e184abad40b4ea3213bacd5">iterations</a>, I managed to hook up view rendering, flash messages, CSRF protection, and email delivery to use Rails instead of Roda, with significantly less code compared to roda-rails. I also managed to make Rodauth code reloadable by inserting a proxy Rack middleware that just calls the Roda app.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_app.rb</span>
<span class="k">class</span> <span class="nc">RodauthApp</span> <span class="o">&lt;</span> <span class="no">Roda</span>
  <span class="n">plugin</span> <span class="ss">:rodauth</span><span class="p">,</span> <span class="ss">csrf: </span><span class="kp">false</span><span class="p">,</span> <span class="ss">flash: </span><span class="kp">false</span> <span class="k">do</span>
    <span class="n">enable</span> <span class="ss">:rails</span><span class="p">,</span> <span class="ss">:login</span><span class="p">,</span> <span class="ss">:create_account</span><span class="p">,</span> <span class="ss">:verify_account</span><span class="p">,</span> <span class="ss">:reset_password</span><span class="p">,</span> <span class="ss">:logout</span>
    <span class="c1"># ... rodauth configuration ...</span>
  <span class="k">end</span>

  <span class="n">route</span> <span class="k">do</span> <span class="o">|</span><span class="n">r</span><span class="o">|</span>
    <span class="n">r</span><span class="p">.</span><span class="nf">rodauth</span> <span class="c1"># handle rodauth requests</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># lib/rodauth/features/rails.rb</span>
<span class="k">module</span> <span class="nn">Rodauth</span>
  <span class="no">Feature</span><span class="p">.</span><span class="nf">define</span><span class="p">(</span><span class="ss">:rails</span><span class="p">,</span> <span class="ss">:Rails</span><span class="p">)</span> <span class="k">do</span>
    <span class="c1"># ... rails integration ...</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># config/initializers/rodauth.rb</span>
<span class="k">class</span> <span class="nc">RodauthMiddleware</span>
  <span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">app</span><span class="p">)</span>
    <span class="vi">@app</span> <span class="o">=</span> <span class="n">app</span>
  <span class="k">end</span>

  <span class="k">def</span> <span class="nf">call</span><span class="p">(</span><span class="n">env</span><span class="p">)</span>
    <span class="no">RodauthApp</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="vi">@app</span><span class="p">).</span><span class="nf">call</span><span class="p">(</span><span class="n">env</span><span class="p">)</span> <span class="c1"># keeps RodauthApp reloadable</span>
  <span class="k">end</span>
<span class="k">end</span>

<span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">middleware</span><span class="p">.</span><span class="nf">use</span> <span class="no">RodauthMiddleware</span>
</code></pre></div></div>

<p>Once I felt things were functioning well enough, I extracted the glue code into the <a href="https://github.com/janko/rodauth-rails">rodauth-rails</a> gem and added tests. I also included an install generator, which created the initial skeleton with sensible default configuration. A new Roda superclass provided a convenience <code class="language-plaintext highlighter-rouge">configure</code> method for loading the Rodauth plugin together with the <code class="language-plaintext highlighter-rouge">rails</code> feature.</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>bundle add rodauth-rails
<span class="nv">$ </span>rails generate rodauth:install
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_app.rb</span>
<span class="k">class</span> <span class="nc">RodauthApp</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">App</span>
  <span class="n">configure</span> <span class="k">do</span> <span class="c1"># automatically loads the rails feature</span>
    <span class="n">enable</span> <span class="ss">:login</span><span class="p">,</span> <span class="ss">:create_account</span><span class="p">,</span> <span class="ss">:verify_account</span><span class="p">,</span> <span class="ss">:reset_password</span><span class="p">,</span> <span class="ss">:logout</span>
    <span class="c1"># ... rodauth configuration ...</span>
  <span class="k">end</span>

  <span class="n">route</span> <span class="k">do</span> <span class="o">|</span><span class="n">r</span><span class="o">|</span>
    <span class="n">r</span><span class="p">.</span><span class="nf">rodauth</span> <span class="c1"># handle rodauth requests</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>By default, Rodauth uses database authentication functions for password matching, which coupled with a <a href="https://github.com/jeremyevans/rodauth#label-PostgreSQL+Database+Setup">two-user database setup</a> allows protecting password hashes even in case of SQL injection (<a href="https://github.com/jeremyevans/rodauth#label-Password+Hash+Access+Via+Database+Functions">read here</a> on how this works). However, I felt this complexity could be daunting when getting started, so I changed the default to do password matching in ruby instead.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">use_database_authentication_functions?</span> <span class="kp">false</span>
<span class="n">account_password_hash_column</span> <span class="ss">:password_hash</span>
</code></pre></div></div>

<p>Rodauth also optionally uses <a href="https://github.com/jeremyevans/rodauth#label-HMAC">HMACs</a> for signing tokens, providing additional security. Since this is quite important, rodauth-rails turns this on by setting the HMAC secret to the Rails’ secret key base.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">hmac_secret</span> <span class="p">{</span> <span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">secret_key_base</span> <span class="p">}</span>
</code></pre></div></div>

<h2 id="active-record">Active Record</h2>

<p>While Rodauth uses Sequel for database interaction, most Rails apps are using Active Record. This meant Rodauth needed to work seamlessly alongside Active Record.</p>

<p>One idea was to develop a Rodauth extension that replaces all Sequel calls with Active Record code. However, given Rodauth’s advanced SQL usage, that would’ve been a monumental effort. It would also have been a huge maintenance burden, since the extension would break with new Rodauth changes, and 3rd-party Rodauth extensions would need to maintain their own Active Record integrations.</p>

<p>So, the initial integration simply connected Sequel to the same database Active Record was connected to, which was then picked up by Rodauth.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">db_config</span> <span class="o">=</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">connection_db_config</span>

<span class="no">Sequel</span><span class="p">.</span><span class="nf">connect</span><span class="p">(</span><span class="ss">adapter: </span><span class="n">db_config</span><span class="p">.</span><span class="nf">adapter</span><span class="p">,</span> <span class="ss">database: </span><span class="n">db_config</span><span class="p">.</span><span class="nf">database</span><span class="p">)</span>
</code></pre></div></div>

<p>However, I noticed that this breaks features such as <a href="https://guides.rubyonrails.org/testing.html#maintaining-the-test-database-schema">maintaining test schema</a>, which requires temporarily disconnecting from the database in order to re-create the test database; even though Active Record connections were closed, Sequel was still holding an open connection, which was blocking database dropping. To address this, I extended Active Record to <a href="https://github.com/janko/rodauth-rails/blob/2b54cba3018d95940fd34af0bc0b17a540b8d2ea/lib/rodauth/rails/active_record_extension.rb">connect &amp; disconnect Sequel in lockstep with Active Record</a>.</p>

<p>This worked, but I quickly encountered a new kind of problem. Because Active Record and Sequel used separate database connections, Active Record code couldn’t reference records created within a Sequel transaction, because to that connection the record simply doesn’t exist (remember, database transactions are tied to a connection).</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Profile</span> <span class="o">&lt;</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span>
  <span class="n">belongs_to</span> <span class="ss">:account</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">RodauthApp</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">App</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># we're still in a Sequel transaction that created the account record</span>
    <span class="n">after_create_account</span> <span class="k">do</span>
      <span class="c1"># creating an associated record with Sequel worked:</span>
      <span class="c1"># db[:profiles].insert(account_id: account_id, name: "New User")</span>

      <span class="c1"># but with Active Record it failed with a foreign key constraint violation:</span>
      <span class="no">Profile</span><span class="p">.</span><span class="nf">create!</span><span class="p">(</span><span class="ss">account_id: </span><span class="n">account_id</span><span class="p">,</span> <span class="ss">name: </span><span class="s2">"New User"</span><span class="p">)</span> <span class="c1"># ~&gt; account record not found</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>My goal was for developers not to have to care that Rodauth uses Sequel, so calling Active Record inside Rodauth should just work. Moreover, <a href="https://github.com/bruno-">Bruno Sutic</a> warned me if Sequel uses a separate database connection, it would mean production databases would have up to twice as many open connections. It became apparent this approach could never achieve the desired developer experience.</p>

<blockquote class="twitter-tweet" data-conversation="none"><p lang="en" dir="ltr">Just browsed the repo. Sequel connection is a bummer.</p>&mdash; Josef Strzibny (@strzibnyj) <a href="https://twitter.com/strzibnyj/status/1253344181650518023?ref_src=twsrc%5Etfw">April 23, 2020</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<p>For the integration to work, I would need to make Sequel reuse Active Record’s database connection. I <a href="https://groups.google.com/g/sequel-talk/c/yQt4ptIDQO4/m/U7yghTFKBAAJ">discussed this idea with Jeremy Evans</a> (the lead Sequel maintainer), and he provided me with some guidance, thanks to which I was able to come up a <a href="https://github.com/janko/sequel-activerecord_connection">solution</a>. It was a Sequel extension that retrieved Active Record connections, kept transaction state and callbacks   synchronized between Sequel and Active Record, integrated SQL instrumentation, and reconciliated adapter differences (see my <a href="https://janko.io/how-i-enabled-sequel-to-reuse-active-record-connection/">previous article</a> for more details).</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>bundle add sequel-activerecord_connection
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span> <span class="o">=</span> <span class="no">Sequel</span><span class="p">.</span><span class="nf">postgres</span><span class="p">(</span><span class="ss">extensions: :activerecord_connection</span><span class="p">)</span>
<span class="no">DB</span><span class="p">[</span><span class="ss">:accounts</span><span class="p">].</span><span class="nf">all</span> <span class="c1"># uses Active Record's database connection</span>
</code></pre></div></div>

<h2 id="model">Model</h2>

<p>Unlike Devise or Sorcery, Rodauth is completely decoupled from models, and any calls need to go through the Rodauth object. If you need to perform Rodauth actions outside of an HTTP request, you can use the <a href="http://rodauth.jeremyevans.net/rdoc/files/doc/internal_request_rdoc.html">internal request</a> feature, which makes an actual Rack call to the Rodauth app:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># performing rodauth actions (class level)</span>
<span class="no">RodauthApp</span><span class="p">.</span><span class="nf">rodauth</span><span class="p">.</span><span class="nf">create_account</span><span class="p">(</span><span class="ss">login: </span><span class="s2">"user@example.com"</span><span class="p">,</span> <span class="ss">password: </span><span class="s2">"secret123"</span><span class="p">)</span>
<span class="no">RodauthApp</span><span class="p">.</span><span class="nf">rodauth</span><span class="p">.</span><span class="nf">verify_account</span><span class="p">(</span><span class="ss">account_login: </span><span class="s2">"user@example.com"</span><span class="p">)</span>

<span class="c1"># calling rodauth methods (instance level)</span>
<span class="n">rodauth</span> <span class="o">=</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="p">.</span><span class="nf">rodauth</span><span class="p">(</span><span class="ss">account_login: </span><span class="s2">"user@example.com"</span><span class="p">)</span>
<span class="n">rodauth</span><span class="p">.</span><span class="nf">get_password_reset_key</span><span class="p">(</span><span class="n">account_id</span><span class="p">)</span> <span class="c1">#=&gt; "DS6dtRNnvzSCWzm8jg4lltOzBE5vTN_xflNdToIPw3A"</span>
<span class="n">rodauth</span><span class="p">.</span><span class="nf">recovery_codes</span> <span class="c1">#=&gt; ["30GRJkr1BheZztvFZcDeRSNy6yhzigXH6zB-yvzP4Io", ...]</span>
</code></pre></div></div>

<p>While I love this decoupling, it would still be nice to be able to at least create accounts and retrieve associations directly through the model. So, I created the <a href="https://github.com/janko/rodauth-model">rodauth-model</a> gem, which provides an interface similar to Active Record’s <code class="language-plaintext highlighter-rouge">has_secure_password</code>, and defines associations based on your Rodauth configuration (together with associated models).</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Account</span> <span class="o">&lt;</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span>
  <span class="kp">include</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="p">.</span><span class="nf">model</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># generating a password hash</span>
<span class="n">account</span> <span class="o">=</span> <span class="no">Account</span><span class="p">.</span><span class="nf">create</span><span class="p">(</span><span class="ss">email: </span><span class="s2">"user@example.com"</span><span class="p">,</span> <span class="ss">password: </span><span class="s2">"secret123"</span><span class="p">)</span>
<span class="n">account</span><span class="p">.</span><span class="nf">password_hash</span> <span class="c1">#=&gt; "$2a$12$k/Ub1I2iomi84RacqY89Hu4.M0vK7klRnRtzorDyvOkVI.hKhkNw."</span>

<span class="n">account</span><span class="p">.</span><span class="nf">password_reset_key</span> <span class="c1">#=&gt; #&lt;Account::PasswordResetKey id: 1, key: "DS6dtRNnvzSCWzm8jg4lltOzBE5vTN_xflNdToIPw3A" ...&gt;</span>
<span class="n">account</span><span class="p">.</span><span class="nf">recovery_codes</span> <span class="c1">#=&gt; [#&lt;Account::RecoveryCode id: 1, code: "30GRJkr1BheZztvFZcDeRSNy6yhzigXH6zB-yvzP4Io"&gt;, ...]</span>
</code></pre></div></div>

<p>Rodauth stores account statuses (unverified, verified, closed) as integers, but we can use <a href="https://api.rubyonrails.org/classes/ActiveRecord/Enum.html"><code class="language-plaintext highlighter-rouge">ActiveRecord::Enum</code></a> to map them to strings, which is what the install generator now does.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Account</span> <span class="o">&lt;</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span>
  <span class="c1"># ...</span>
  <span class="n">enum</span> <span class="ss">:status</span><span class="p">,</span> <span class="ss">unverified: </span><span class="mi">1</span><span class="p">,</span> <span class="ss">verified: </span><span class="mi">2</span><span class="p">,</span> <span class="ss">closed: </span><span class="mi">3</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">account</span> <span class="o">=</span> <span class="no">Account</span><span class="p">.</span><span class="nf">find</span><span class="p">(</span><span class="mi">123</span><span class="p">)</span>
<span class="n">account</span><span class="p">.</span><span class="nf">status</span> <span class="c1">#=&gt; "unverified"</span>

<span class="n">account</span><span class="p">.</span><span class="nf">verified?</span> <span class="c1">#=&gt; false</span>
<span class="n">account</span><span class="p">.</span><span class="nf">status</span> <span class="o">=</span> <span class="s2">"verified"</span>
<span class="n">account</span><span class="p">.</span><span class="nf">verified?</span> <span class="c1">#=&gt; true</span>

<span class="no">Account</span><span class="p">.</span><span class="nf">closed</span> <span class="c1">#=&gt; [#&lt;Account id: 456, status: "closed" ...&gt;, ...]</span>
</code></pre></div></div>

<h2 id="routes-introspection">Routes introspection</h2>

<p>In this architecture, Rodauth routes are handled by the Rack middleware that’s sitting in front of the Rails router. This has the benefit of allowing you to perform authentication actions such as requiring authentication, checking active sessions, and remembering from cookie in a single place, thus better encapsulating authentication logic.</p>

<p>However, since Rodauth routes aren’t registered within the Rails router, they don’t show up in <code class="language-plaintext highlighter-rouge">rails routes</code>. Rails’ routes introspection doesn’t currently have capability of registering custom routes, and Roda’s routing is <a href="https://janko.io/introduction-to-roda/">dynamic</a>, so it’s not possible to retrieve its routes programmatically.</p>

<p>Eventually I managed to implement a <a href="https://github.com/janko/rodauth-rails/blob/169e9e06bf9cf0dc4ecfe669af2f7f542bf5ba63/lib/rodauth/rails/tasks.rake"><code class="language-plaintext highlighter-rouge">rodauth:routes</code></a> Rake task, which retrieves the route paths from the Rodauth app, and parses the source code for HTTP verbs. It’s not ideal, but it should be good enough.</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails rodauth:routes
<span class="c"># Routes handled by RodauthApp:</span>
<span class="c"># </span>
<span class="c">#   GET/POST  /login                   rodauth.login_path</span>
<span class="c">#   GET/POST  /create-account          rodauth.create_account_path</span>
<span class="c">#   POST      /email-auth-request      rodauth.email_auth_request_path</span>
<span class="c">#   GET/POST  /email-auth              rodauth.email_auth_path</span>
<span class="c">#   GET/POST  /logout                  rodauth.logout_path</span>
<span class="c">#   GET/POST  /reset-password-request  rodauth.reset_password_request_path</span>
<span class="c">#   GET/POST  /reset-password          rodauth.reset_password_path</span>
<span class="c">#   GET/POST  /change-password         rodauth.change_password_path</span>
<span class="c">#   GET/POST  /change-login            rodauth.change_login_path</span>
<span class="c">#   GET/POST  /verify-login-change     rodauth.verify_login_change_path</span>
<span class="c">#   GET/POST  /close-account           rodauth.close_account_path</span>
<span class="c">#   GET/POST  /verify-account-resend   rodauth.verify_account_resend_path</span>
<span class="c">#   GET/POST  /verify-account          rodauth.verify_account_path</span>
<span class="c"># </span>
<span class="c">#   GET       /admin/multifactor-manage      rodauth(:admin).two_factor_manage_path</span>
<span class="c">#   GET       /admin/multifactor-auth        rodauth(:admin).two_factor_auth_path</span>
<span class="c">#   GET/POST  /admin/multifactor-disable     rodauth(:admin).two_factor_disable_path</span>
<span class="c">#   GET/POST  /admin/otp-auth                rodauth(:admin).otp_auth_path</span>
<span class="c">#   GET/POST  /admin/otp-setup               rodauth(:admin).otp_setup_path</span>
<span class="c">#   GET/POST  /admin/otp-disable             rodauth(:admin).otp_disable_path</span>
<span class="c">#   GET/POST  /admin/recovery-auth           rodauth(:admin).recovery_auth_path</span>
<span class="c">#   GET/POST  /admin/recovery-codes          rodauth(:admin).recovery_codes_path</span>
<span class="c">#   POST      /admin/unlock-account-request  rodauth(:admin).unlock_account_request_path</span>
<span class="c">#   GET/POST  /admin/unlock-account          rodauth(:admin).unlock_account_path</span>
</code></pre></div></div>

<h2 id="generators">Generators</h2>

<p>Before working on this gem, I didn’t realize how important code generators are for convenience, and also how difficult they are to get right. There are currently three generators rodauth-rails ships with:</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">rodauth:install</code> – sets up initial skeleton with default auth features and migrations</li>
  <li><code class="language-plaintext highlighter-rouge">rodauth:views</code> – imports ERB view templates for customization</li>
  <li><code class="language-plaintext highlighter-rouge">rodauth:migration</code> – generates migrations for selected authentication features</li>
</ul>

<p>The generators needed to handle various scenarios, such as RSpec vs Minitest, fixtures vs factory_bot, Active Record vs Sequel, different SQL adapters, API-only mode, UUID primary keys, and more.</p>

<h3 id="schema-migrations">Schema migrations</h3>

<p>I wanted to remove any Sequel code from sight, so I translated <a href="http://rodauth.jeremyevans.net/rdoc/files/README_rdoc.html#label-Creating+tables">Sequel migrations</a> into Active Record code, and generated those in Active Record migrations. If Sequel happens to be used as the main database library, we’d generate Sequel migrations instead.</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate rodauth:migration otp active_sessions
<span class="c"># create  db/migrate/20221012110706_create_rodauth_otp_active_sessions.rb</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">CreateRodauthOtpActiveSessions</span> <span class="o">&lt;</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Migration</span><span class="p">[</span><span class="mf">7.0</span><span class="p">]</span>
  <span class="k">def</span> <span class="nf">change</span>
    <span class="c1"># Used by the otp feature</span>
    <span class="n">create_table</span> <span class="ss">:account_otp_keys</span> <span class="k">do</span> <span class="o">|</span><span class="n">t</span><span class="o">|</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">foreign_key</span> <span class="ss">:accounts</span><span class="p">,</span> <span class="ss">column: :id</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">string</span> <span class="ss">:key</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">integer</span> <span class="ss">:num_failures</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span><span class="p">,</span> <span class="ss">default: </span><span class="mi">0</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">datetime</span> <span class="ss">:last_use</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span><span class="p">,</span> <span class="ss">default: </span><span class="o">-&gt;</span> <span class="p">{</span> <span class="s2">"CURRENT_TIMESTAMP"</span> <span class="p">}</span>
    <span class="k">end</span>

    <span class="c1"># Used by the active sessions feature</span>
    <span class="n">create_table</span> <span class="ss">:account_active_session_keys</span><span class="p">,</span> <span class="ss">primary_key: </span><span class="p">[</span><span class="ss">:account_id</span><span class="p">,</span> <span class="ss">:session_id</span><span class="p">]</span> <span class="k">do</span> <span class="o">|</span><span class="n">t</span><span class="o">|</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">references</span> <span class="ss">:account</span><span class="p">,</span> <span class="ss">foreign_key: </span><span class="kp">true</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">string</span> <span class="ss">:session_id</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">datetime</span> <span class="ss">:created_at</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span><span class="p">,</span> <span class="ss">default: </span><span class="o">-&gt;</span> <span class="p">{</span> <span class="s2">"CURRENT_TIMESTAMP"</span> <span class="p">}</span>
      <span class="n">t</span><span class="p">.</span><span class="nf">datetime</span> <span class="ss">:last_use</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span><span class="p">,</span> <span class="ss">default: </span><span class="o">-&gt;</span> <span class="p">{</span> <span class="s2">"CURRENT_TIMESTAMP"</span> <span class="p">}</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<h3 id="view-templates">View templates</h3>

<p>The <a href="https://github.com/jeremyevans/rodauth/tree/master/templates">built-in view templates</a> use <a href="https://github.com/rtomayko/tilt">Tilt</a>’s interpolated string engine, which avoids ERB dependency, but requires work to adapt for Rails. So, rodauth-rails’ views generator imports already converted <a href="https://github.com/janko/rodauth-rails/tree/2b54cba3018d95940fd34af0bc0b17a540b8d2ea/lib/generators/rodauth/templates/app/views/rodauth">ERB view templates</a> that use familiar Rails’ form helpers.</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate rodauth:views login create_account lockout 
<span class="c"># create  app/views/rodauth/_login_form.html.erb</span>
<span class="c"># create  app/views/rodauth/_login_form_footer.html.erb</span>
<span class="c"># create  app/views/rodauth/_login_form_header.html.erb</span>
<span class="c"># create  app/views/rodauth/login.html.erb</span>
<span class="c"># create  app/views/rodauth/multi_phase_login.html.erb</span>
<span class="c"># create  app/views/rodauth/create_account.html.erb</span>
<span class="c"># create  app/views/rodauth/unlock_account_request.html.erb</span>
<span class="c"># create  app/views/rodauth/unlock_account.html.erb</span>
</code></pre></div></div>
<div class="language-erb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;%=</span> <span class="n">form_with</span> <span class="ss">url: </span><span class="n">rodauth</span><span class="p">.</span><span class="nf">unlock_account_path</span><span class="p">,</span> <span class="ss">method: :post</span><span class="p">,</span> <span class="ss">data: </span><span class="p">{</span> <span class="ss">turbo: </span><span class="kp">false</span> <span class="p">}</span> <span class="k">do</span> <span class="o">|</span><span class="n">form</span><span class="o">|</span> <span class="cp">%&gt;</span>
  <span class="cp">&lt;%=</span><span class="o">=</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">unlock_account_explanatory_text</span> <span class="cp">%&gt;</span>

  <span class="cp">&lt;%</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">unlock_account_requires_password?</span> <span class="cp">%&gt;</span>
    <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"mb-3"</span><span class="nt">&gt;</span>
      <span class="cp">&lt;%=</span> <span class="n">form</span><span class="p">.</span><span class="nf">label</span> <span class="s2">"password"</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">password_label</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"form-label"</span> <span class="cp">%&gt;</span>
      <span class="cp">&lt;%=</span> <span class="n">form</span><span class="p">.</span><span class="nf">password_field</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">password_param</span><span class="p">,</span> <span class="ss">value: </span><span class="s2">""</span><span class="p">,</span> <span class="ss">id: </span><span class="s2">"password"</span><span class="p">,</span> <span class="ss">autocomplete: </span><span class="n">rodauth</span><span class="p">.</span><span class="nf">password_field_autocomplete_value</span><span class="p">,</span> <span class="ss">required: </span><span class="kp">true</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"form-control </span><span class="si">#{</span><span class="s2">"is-invalid"</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="n">rodauth</span><span class="p">.</span><span class="nf">password_param</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span> <span class="ss">aria: </span><span class="p">({</span> <span class="ss">invalid: </span><span class="kp">true</span><span class="p">,</span> <span class="ss">describedby: </span><span class="s2">"password_error_message"</span> <span class="p">}</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="n">rodauth</span><span class="p">.</span><span class="nf">password_param</span><span class="p">))</span> <span class="cp">%&gt;</span>
      <span class="cp">&lt;%=</span> <span class="n">content_tag</span><span class="p">(</span><span class="ss">:span</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="n">rodauth</span><span class="p">.</span><span class="nf">password_param</span><span class="p">),</span> <span class="ss">class: </span><span class="s2">"invalid-feedback"</span><span class="p">,</span> <span class="ss">id: </span><span class="s2">"password_error_message"</span><span class="p">)</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="n">rodauth</span><span class="p">.</span><span class="nf">password_param</span><span class="p">)</span> <span class="cp">%&gt;</span>
    <span class="nt">&lt;/div&gt;</span>
  <span class="cp">&lt;%</span> <span class="k">end</span> <span class="cp">%&gt;</span>

  <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"mb-3"</span><span class="nt">&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">form</span><span class="p">.</span><span class="nf">submit</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">unlock_account_button</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"btn btn-primary"</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;/div&gt;</span>
<span class="cp">&lt;%</span> <span class="k">end</span> <span class="cp">%&gt;</span>
</code></pre></div></div>

<p>Because not all Rodauth actions are Turbo-compatible (multi-phase login and viewing recovery codes return a 200 response on form submits), I have chosen to disable Turbo by default for all HTML forms, to ensure everything works.</p>

<h2 id="future-plans">Future plans</h2>

<ul>
  <li>
    <p>I would like the migration generator to support database authentication functions, to make it easier for developers to better secure their password hashes. I have been working on it on &amp; off, but it’s fairly complex to generate correct migration code, especially since different SQL databases (PostgreSQL, MySQL, SQL Server) require different setups.</p>
  </li>
  <li>
    <p>Default Rodauth view templates use Bootstrap markup, but Ben Koshy has been working on adding <a href="https://github.com/janko/rodauth-rails/pull/114">Tailwind CSS support</a>, which will be a nice addition. You’ll be able to pass <code class="language-plaintext highlighter-rouge">--css=tailwind</code>, which will be the default when the <code class="language-plaintext highlighter-rouge">tailwindcss-rails</code> gem is used.</p>
  </li>
  <li>
    <p>I want to make it easier for 3rd-party Rodauth extensions to provide migrations/views for Rails generators. The <a href="https://github.com/HoneyryderChuck/rodauth-oauth">rodauth-oauth</a> gem currently provides its own generators (<code class="language-plaintext highlighter-rouge">rodauth:oauth:install</code> and <code class="language-plaintext highlighter-rouge">rodauth:oauth:views</code>), but it would be nice if they didn’t have to be duplicated.</p>
  </li>
  <li>
    <p>Rodauth methods are called through the Rodauth object, and you’re encouraged to define convenience controller/view helpers that suit your application’s needs. That being said, having some default helpers <a href="https://github.com/heartcombo/devise#controller-filters-and-helpers">like Devise has</a> (and even something like <a href="https://github.com/heartcombo/devise/blob/6d32d2447cc0f3739d9732246b5a5bde98d9e032/lib/devise/controllers/helpers.rb#L18-L38">Devise groups</a>) probably might go a long way. I was also considering more convenient routing helpers, perhaps even a builder for defining custom ones.</p>

    <div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># config/routes.rb</span>
<span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">routes</span><span class="p">.</span><span class="nf">draw</span> <span class="k">do</span>
  <span class="c1"># this is what we have today</span>
  <span class="n">constraints</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="p">.</span><span class="nf">authenticated</span> <span class="k">do</span>
    <span class="c1"># ...</span>
  <span class="k">end</span>

  <span class="c1"># but maybe Devise syntax is worthwhile</span>
  <span class="n">authenticate</span> <span class="k">do</span>
    <span class="c1"># ...</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div>    </div>
  </li>
</ul>

<h2 id="closing-words">Closing words</h2>

<p>There are many more things the Rails integration takes care of, such as ignoring asset requests from Sprockets or Propshaft, instrumentation/logging of Rodauth requests, executing controller callbacks &amp; rescue handlers, handling background emails, testing helpers, and Rails 4.2+ &amp; JRuby support. But I think I rambled on for long enough.</p>

<p>The purpose for this article wasn’t to bolster on my achievements, but to share my realization about the price of convenience, and how much work it can actually take integrate a generic library into Rails, especially when it does the work of a Rails engine. I cannot thank Jeremy Evans enough for discussing, reviewing, and merging <a href="https://github.com/jeremyevans/rodauth/pulls?q=is%3Apr+author%3Ajanko">every one of my additions</a> to Rodauth that helped enable a smooth Rails integration :pray:</p>

<p>I hope I fulfilled my goal of making Rodauth easy to work with in Rails. That being said, I’m grateful for the continuous feedback I’m getting from people that are using Rodauth. I’m trying to convert many of my answers into a <a href="https://github.com/janko/rodauth-rails/wiki">wiki page</a>, a <a href="https://rodauth.jeremyevans.net/documentation.html#guides">how-to guide</a> or a <a href="https://www.youtube.com/user/Junky098">screencast</a>, to grow the knowledge around handling common use cases.</p>]]></content><author><name>Janko Marohnić</name><email>janko@hey.com</email></author><category term="rodauth" /><summary type="html"><![CDATA[When Rodauth came out, I was excited to finally have a full-featured authentication framework that wasn’t tied to Rails, given that existing solutions required either Rails (Devise, Sorcery), or at least Active Record (Authlogic). Even though I mainly develop in Rails, I want other Ruby web frameworks to be viable alternatives, so I’m naturally drawn to generic solutions that everyone can use.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://janko.io/images/social.jpg" /><media:content medium="image" url="https://janko.io/images/social.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">How I Enabled Sequel to Reuse Active Record’s Database Connection</title><link href="https://janko.io/how-i-enabled-sequel-to-reuse-active-record-connection/" rel="alternate" type="text/html" title="How I Enabled Sequel to Reuse Active Record’s Database Connection" /><published>2022-04-24T00:00:00+00:00</published><updated>2022-04-24T00:00:00+00:00</updated><id>https://janko.io/how-i-enabled-sequel-to-reuse-active-record-connection</id><content type="html" xml:base="https://janko.io/how-i-enabled-sequel-to-reuse-active-record-connection/"><![CDATA[<p>When I started developing the <a href="https://github.com/janko/rodauth-rails">Rails integration for Rodauth</a>, one of the first problems I needed to solve was how to make Rodauth work seamlessly with Active Record, given that it uses <a href="https://sequel.jeremyevans.net/">Sequel</a> for database interaction. I believed these two could coexist together, because Sequel is mostly hidden from the Rodauth user anyway, and all that really matters is that Rodauth’s SQL statements get executed on the database.</p>

<p>My first approach was to have Sequel connect to the same database as Active Record, and create and close connections in <a href="https://github.com/janko/rodauth-rails/blob/2b54cba3018d95940fd34af0bc0b17a540b8d2ea/lib/rodauth/rails/active_record_extension.rb">lockstep with Active Record</a>, so that Sequel is connected to the database if and only if Active Record is too. While this was functional, it didn’t play well with transactions. Some of Rodauth’s configuration blocks are executed within a Sequel transaction, and I wanted developers to be able to call Active Record inside them, and have it just work. But this wasn’t the case, because it turns out, if you have two different connections, they aren’t aware of each other’s transactions; you might as well be using two different processes.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">establish_connection</span><span class="p">(</span><span class="s2">"postgresql:///mydb"</span><span class="p">)</span>
<span class="no">DB</span> <span class="o">=</span> <span class="no">Sequel</span><span class="p">.</span><span class="nf">connect</span><span class="p">(</span><span class="s2">"postgresql:///mydb"</span><span class="p">)</span>

<span class="k">class</span> <span class="nc">Account</span> <span class="o">&lt;</span> <span class="no">Sequel</span><span class="o">::</span><span class="no">Model</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">Profile</span> <span class="o">&lt;</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span>
<span class="k">end</span>

<span class="c1"># open sequel transaction</span>
<span class="no">DB</span><span class="p">.</span><span class="nf">transaction</span> <span class="k">do</span>
  <span class="c1"># create record using sequel connection</span>
  <span class="n">account</span> <span class="o">=</span> <span class="no">Account</span><span class="p">.</span><span class="nf">create</span><span class="p">(</span><span class="ss">email: </span><span class="s2">"user@example.com"</span><span class="p">,</span> <span class="ss">password_hash: </span><span class="s2">"..."</span><span class="p">)</span>

  <span class="c1"># open active record transaction</span>
  <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">transaction</span> <span class="k">do</span>
    <span class="c1"># create associates record using active record connection</span>
    <span class="no">Profile</span><span class="p">.</span><span class="nf">create</span><span class="p">(</span><span class="ss">name: </span><span class="s2">"User"</span><span class="p">,</span> <span class="ss">account_id: </span><span class="n">account</span><span class="p">.</span><span class="nf">id</span><span class="p">)</span>
    <span class="c1">#~&gt; foreign key constraint violation: given account_id is not present in table "accounts"</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>The example above fails because, while the account record was created by Sequel’s connection, its transaction hasn’t yet been committed at the time Active Record’s connection attempted to create the associated profile record. Even though Active Record’s transaction block is physically nested inside Sequel’s, transactions are tied to connections that opened them, so as far as the database is concerned these are two independent transactions.</p>

<p>Moreover, if Sequel did use its own database connection, that would mean the number of open connections to the database would double, which could impact performance and hit maximum connection limits. So, I knew I needed to find a way to make Sequel reuse Active Record’s database connection instead of creating its own.</p>

<p>I decided to build this as a <a href="https://sequel.jeremyevans.net/rdoc/files/doc/extensions_rdoc.html#label-Database+Extensions">database extension</a> for Sequel, which when loaded, switches Sequel to use Active Record’s database connection:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Sequel</span>
  <span class="k">class</span> <span class="nc">ActiveRecordConnection</span>
    <span class="c1"># ... database overrides ...</span>
  <span class="k">end</span>

  <span class="no">Database</span><span class="p">.</span><span class="nf">register_extension</span><span class="p">(</span><span class="ss">:activerecord_connection</span><span class="p">,</span> <span class="no">ActiveRecordConnection</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span> <span class="o">=</span> <span class="no">Sequel</span><span class="p">.</span><span class="nf">connect</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
<span class="no">DB</span><span class="p">.</span><span class="nf">extension</span> <span class="ss">:activerecord_connection</span>
<span class="no">DB</span><span class="p">.</span><span class="nf">run</span> <span class="s2">"SELECT ..."</span> <span class="c1"># executed on Active Record's database connection</span>
</code></pre></div></div>

<h2 id="reusing-the-connection">Reusing the connection</h2>

<p>Sequel’s connection pool is the one in charge of creating new database connections when they’re needed. So, the first and most important step was to bypass it, and retrieve Active Record’s database connection instead.</p>

<p>The Sequel connection is retrieved in <code class="language-plaintext highlighter-rouge">Database#synchronize</code>, which gets called for every query, so that’s the ideal place to put the override:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Sequel::ActiveRecordConnection</span>
  <span class="k">def</span> <span class="nf">synchronize</span><span class="p">(</span><span class="o">*</span><span class="p">)</span>
    <span class="k">yield</span> <span class="n">activerecord_connection</span><span class="p">.</span><span class="nf">raw_connection</span>
  <span class="k">end</span>

  <span class="kp">private</span>

  <span class="k">def</span> <span class="nf">activerecord_connection</span>
    <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">connection</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">.</span><span class="nf">synchronize</span> <span class="k">do</span> <span class="o">|</span><span class="n">connection</span><span class="o">|</span>
  <span class="n">connection</span> <span class="c1"># Active Record's connection object</span>
<span class="k">end</span>
</code></pre></div></div>

<p>I also needed to account for the fact that Sequel adapters use different connection options than Active Record, and store prepared statements in the connection object. For <a href="https://github.com/janko/sequel-activerecord_connection/blob/ee2039e2a1d297f70c0a7d7adab7aff47c52084b/lib/sequel/extensions/activerecord_connection/sqlite.rb">SQLite</a> and <a href="https://github.com/janko/sequel-activerecord_connection/blob/ee2039e2a1d297f70c0a7d7adab7aff47c52084b/lib/sequel/extensions/activerecord_connection/mysql2.rb">MySQL</a> handling this required relatively little code, while for <a href="https://github.com/janko/sequel-activerecord_connection/blob/ee2039e2a1d297f70c0a7d7adab7aff47c52084b/lib/sequel/extensions/activerecord_connection/postgres.rb">PostgreSQL</a> I unfortunately needed to copy-paste methods defined on Sequel adapter’s subclass of <code class="language-plaintext highlighter-rouge">PG::Connection</code>.</p>

<h2 id="syncing-transaction-state">Syncing transaction state</h2>

<p>Both Sequel and Active Record track which transactions are in progress and in which order. Currently, even though the connection is shared, Sequel doesn’t know about transactions opened by Active Record and vice-versa.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">.</span><span class="nf">transaction</span> <span class="k">do</span>
  <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">connection</span><span class="p">.</span><span class="nf">open_transactions</span> <span class="c1">#=&gt; 0</span>
<span class="k">end</span>

<span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">transaction</span> <span class="k">do</span>
  <span class="no">DB</span><span class="p">.</span><span class="nf">in_transaction?</span> <span class="c1">#=&gt; false</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Syncing this state is important for handling nested transaction blocks, which should either reuse the outer transaction or use a savepoint, depending on the setup. Sequel saves informations about transactions for each connection in <code class="language-plaintext highlighter-rouge">@transactions</code> instance variable:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">.</span><span class="nf">transaction</span><span class="p">(</span><span class="ss">auto_savepoint: </span><span class="kp">true</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">conn</span><span class="o">|</span>
  <span class="no">DB</span><span class="p">.</span><span class="nf">instance_variable_get</span><span class="p">(</span><span class="ss">:@transactions</span><span class="p">)[</span><span class="n">conn</span><span class="p">]</span> <span class="c1">#=&gt; {savepoints: [{auto_savepoint: true}]}</span>

  <span class="no">DB</span><span class="p">.</span><span class="nf">transaction</span> <span class="k">do</span> <span class="c1"># creates a savepoint</span>
    <span class="no">DB</span><span class="p">.</span><span class="nf">instance_variable_get</span><span class="p">(</span><span class="ss">:@transactions</span><span class="p">)[</span><span class="n">conn</span><span class="p">]</span> <span class="c1">#=&gt; {savepoints: [{auto_savepoint: true}, {auto_savepoint: nil}]}</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>We’ll override the method accessing this instance variable, and sync state about any transactions opened by Active Record:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Sequel::ActiveRecordConnection</span>
  <span class="c1"># ...</span>
  <span class="kp">private</span>

  <span class="k">def</span> <span class="nf">_trans</span><span class="p">(</span><span class="n">conn</span><span class="p">)</span>
    <span class="nb">hash</span> <span class="o">=</span> <span class="k">super</span> <span class="o">||</span> <span class="p">{</span> <span class="ss">savepoints: </span><span class="p">[],</span> <span class="ss">activerecord: </span><span class="kp">true</span> <span class="p">}</span>

    <span class="c1"># add any transactions/savepoints opened via Active Record</span>
    <span class="k">while</span> <span class="nb">hash</span><span class="p">[</span><span class="ss">:savepoints</span><span class="p">].</span><span class="nf">length</span> <span class="o">&lt;</span> <span class="n">activerecord_connection</span><span class="p">.</span><span class="nf">open_transactions</span>
      <span class="nb">hash</span><span class="p">[</span><span class="ss">:savepoints</span><span class="p">]</span> <span class="o">&lt;&lt;</span> <span class="p">{</span> <span class="ss">activerecord: </span><span class="kp">true</span> <span class="p">}</span>
    <span class="k">end</span>
    <span class="c1"># remove any transactions/savepoints closed via Active Record</span>
    <span class="k">while</span> <span class="nb">hash</span><span class="p">[</span><span class="ss">:savepoints</span><span class="p">].</span><span class="nf">length</span> <span class="o">&gt;</span> <span class="n">activerecord_connection</span><span class="p">.</span><span class="nf">open_transactions</span> <span class="o">&amp;&amp;</span> <span class="nb">hash</span><span class="p">[</span><span class="ss">:savepoints</span><span class="p">].</span><span class="nf">last</span><span class="p">[</span><span class="ss">:activerecord</span><span class="p">]</span>
      <span class="nb">hash</span><span class="p">[</span><span class="ss">:savepoints</span><span class="p">].</span><span class="nf">pop</span>
    <span class="k">end</span>
    <span class="c1"># sync knowledge about joinability of current Active Record transaction/savepoint</span>
    <span class="k">if</span> <span class="n">activerecord_connection</span><span class="p">.</span><span class="nf">transaction_open?</span> <span class="o">&amp;&amp;</span> <span class="o">!</span><span class="n">activerecord_connection</span><span class="p">.</span><span class="nf">current_transaction</span><span class="p">.</span><span class="nf">joinable?</span>
      <span class="nb">hash</span><span class="p">[</span><span class="ss">:savepoints</span><span class="p">].</span><span class="nf">last</span><span class="p">[</span><span class="ss">:auto_savepoint</span><span class="p">]</span> <span class="o">=</span> <span class="kp">true</span>
    <span class="k">end</span>

    <span class="k">if</span> <span class="nb">hash</span><span class="p">[</span><span class="ss">:savepoints</span><span class="p">].</span><span class="nf">empty?</span> <span class="o">&amp;&amp;</span> <span class="nb">hash</span><span class="p">[</span><span class="ss">:activerecord</span><span class="p">]</span> <span class="c1"># Active Record closed last transaction</span>
      <span class="no">Sequel</span><span class="p">.</span><span class="nf">synchronize</span> <span class="p">{</span> <span class="vi">@transactions</span><span class="p">.</span><span class="nf">delete</span><span class="p">(</span><span class="n">conn</span><span class="p">)</span> <span class="p">}</span>
    <span class="k">else</span>
      <span class="no">Sequel</span><span class="p">.</span><span class="nf">synchronize</span> <span class="p">{</span> <span class="vi">@transactions</span><span class="p">[</span><span class="n">conn</span><span class="p">]</span> <span class="o">=</span> <span class="nb">hash</span> <span class="p">}</span>
    <span class="k">end</span>

    <span class="k">super</span>
  <span class="k">end</span>
  <span class="c1"># ...</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">transaction</span><span class="p">(</span><span class="ss">requires_new: </span><span class="kp">true</span><span class="p">)</span> <span class="k">do</span>
  <span class="no">DB</span><span class="p">.</span><span class="nf">in_transaction?</span> <span class="c1">#=&gt; true</span>
  <span class="no">DB</span><span class="p">.</span><span class="nf">transaction</span> <span class="k">do</span>
    <span class="no">DB</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="ss">:in_savepoint?</span><span class="p">)</span> <span class="c1">#=&gt; true</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>This takes care of Sequel state, now we need to ensure that Active Record’s state is updated when opening transactions via Sequel. We can do this by calling Active Record for beginning, committing, and rolling back transactions:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Sequel::ActiveRecordConnection</span>
  <span class="c1"># ...</span>
  <span class="kp">private</span>

  <span class="k">def</span> <span class="nf">begin_transaction</span><span class="p">(</span><span class="n">conn</span><span class="p">,</span> <span class="n">opts</span> <span class="o">=</span> <span class="no">OPTS</span><span class="p">)</span>
    <span class="n">activerecord_connection</span><span class="p">.</span><span class="nf">begin_transaction</span><span class="p">(</span><span class="ss">joinable: </span><span class="o">!</span><span class="n">opts</span><span class="p">[</span><span class="ss">:auto_savepoint</span><span class="p">])</span>
  <span class="k">end</span>

  <span class="k">def</span> <span class="nf">commit_transaction</span><span class="p">(</span><span class="n">conn</span><span class="p">,</span> <span class="n">opts</span> <span class="o">=</span> <span class="no">OPTS</span><span class="p">)</span>
    <span class="n">activerecord_connection</span><span class="p">.</span><span class="nf">commit_transaction</span>
  <span class="k">end</span>

  <span class="k">def</span> <span class="nf">rollback_transaction</span><span class="p">(</span><span class="n">conn</span><span class="p">,</span> <span class="n">opts</span> <span class="o">=</span> <span class="no">OPTS</span><span class="p">)</span>
    <span class="n">activerecord_connection</span><span class="p">.</span><span class="nf">rollback_transaction</span>
  <span class="k">end</span>
  <span class="c1"># ...</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">.</span><span class="nf">transaction</span><span class="p">(</span><span class="ss">auto_savepoint: </span><span class="kp">true</span><span class="p">)</span> <span class="k">do</span>
  <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">connection</span><span class="p">.</span><span class="nf">open_transactions</span> <span class="c1">#=&gt; 1</span>
  <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">transaction</span> <span class="k">do</span>
    <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">connection</span><span class="p">.</span><span class="nf">open_transactions</span> <span class="c1">#=&gt; 2</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<h2 id="fixing-transaction-hooks">Fixing transaction hooks</h2>

<p>Because we’ve preserved the format of transaction state, Sequel’s after commit/rollback hooks still work when Sequel holds the outer transaction:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">.</span><span class="nf">transaction</span> <span class="k">do</span>
  <span class="no">DB</span><span class="p">.</span><span class="nf">after_commit</span> <span class="p">{</span> <span class="nb">puts</span> <span class="s2">"=&gt; after commit"</span> <span class="p">}</span>
  <span class="no">DB</span><span class="p">.</span><span class="nf">after_rollback</span> <span class="p">{</span> <span class="nb">puts</span> <span class="s2">"=&gt; after rollback"</span> <span class="p">}</span>
  <span class="no">DB</span><span class="p">.</span><span class="nf">run</span> <span class="s2">"SELECT 1"</span>
<span class="k">end</span>
<span class="c1"># BEGIN</span>
<span class="c1"># SELECT 1</span>
<span class="c1"># COMMIT</span>
<span class="c1"># =&gt; after commit</span>
</code></pre></div></div>

<p>However, they don’t work when Active Record holds the outer transaction, because in that case Active Record is the one committing the transaction, and Sequel doesn’t get notified.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">transaction</span> <span class="k">do</span>
  <span class="no">DB</span><span class="p">.</span><span class="nf">after_commit</span> <span class="p">{</span> <span class="nb">puts</span> <span class="s2">"doesn't get called"</span> <span class="p">}</span>
  <span class="no">DB</span><span class="p">.</span><span class="nf">run</span> <span class="s2">"SELECT 1"</span>
<span class="k">end</span>
<span class="c1"># BEGIN</span>
<span class="c1"># SELECT 1</span>
<span class="c1"># COMMIT</span>
</code></pre></div></div>

<p>We can fix this by using the <a href="https://github.com/Envek/after_commit_everywhere">after_commit_everywhere</a> gem to register after commit/rollback callbacks into Active Record when it holds the outer transaction. Sequel will call either <code class="language-plaintext highlighter-rouge">#add_transaction_hook</code> or <code class="language-plaintext highlighter-rouge">#add_savepoint_hook</code> method, depending on whether the hook was registered within a transaction or a savepoint, so we’ll override those:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>gem <span class="nb">install </span>after_commit_everywhere
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s2">"after_commit_everywhere"</span>

<span class="k">class</span> <span class="nc">Sequel::ActiveRecordConnection</span>
  <span class="c1"># ...</span>
  <span class="kp">private</span>

  <span class="k">def</span> <span class="nf">add_transaction_hook</span><span class="p">(</span><span class="n">conn</span><span class="p">,</span> <span class="n">type</span><span class="p">,</span> <span class="n">block</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">_trans</span><span class="p">(</span><span class="n">conn</span><span class="p">)[</span><span class="ss">:activerecord</span><span class="p">]</span> <span class="c1"># Active Record holds the outer transaction</span>
      <span class="no">AfterCommitEverywhere</span><span class="p">.</span><span class="nf">public_send</span><span class="p">(</span><span class="n">type</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">block</span><span class="p">)</span>
    <span class="k">else</span>
      <span class="k">super</span>
    <span class="k">end</span>
  <span class="k">end</span>

  <span class="k">def</span> <span class="nf">add_savepoint_hook</span><span class="p">(</span><span class="n">conn</span><span class="p">,</span> <span class="n">type</span><span class="p">,</span> <span class="n">block</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">_trans</span><span class="p">(</span><span class="n">conn</span><span class="p">)[</span><span class="ss">:savepoints</span><span class="p">].</span><span class="nf">last</span><span class="p">[</span><span class="ss">:activerecord</span><span class="p">]</span> <span class="c1"># Active Record holds the savepoint</span>
      <span class="no">AfterCommitEverywhere</span><span class="p">.</span><span class="nf">public_send</span><span class="p">(</span><span class="n">type</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">block</span><span class="p">)</span>
    <span class="k">else</span>
      <span class="k">super</span>
    <span class="k">end</span>
  <span class="k">end</span>
  <span class="c1"># ...</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">transaction</span> <span class="k">do</span>
  <span class="no">DB</span><span class="p">.</span><span class="nf">after_commit</span> <span class="p">{</span> <span class="nb">puts</span> <span class="s2">"=&gt; gets called"</span> <span class="p">}</span>
  <span class="no">DB</span><span class="p">.</span><span class="nf">run</span> <span class="s2">"SELECT 1"</span>
<span class="k">end</span>
<span class="c1"># BEGIN</span>
<span class="c1"># SELECT 1</span>
<span class="c1"># COMMIT</span>
<span class="c1"># =&gt; gets called</span>
</code></pre></div></div>

<h2 id="instrumenting-sql-queries">Instrumenting SQL queries</h2>

<p>Active Record logs its SQL queries through a <a href="https://github.com/rails/rails/blob/main/activerecord/lib/active_record/log_subscriber.rb">log subscriber</a> that listens for <code class="language-plaintext highlighter-rouge">sql.active_record</code> notifications. To make the integration seamless, I wanted Sequel’s SQL queries to be logged via Active Record’s logger.</p>

<p>Sequel logging is happening in <code class="language-plaintext highlighter-rouge">Database#log_connection_yield</code>, so we’ll want to override that, and instrument the query execution with Active Support:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Sequel::ActiveRecordConnection</span>
  <span class="c1"># ...</span>
  <span class="k">def</span> <span class="nf">log_connection_yield</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">conn</span><span class="p">,</span> <span class="n">args</span> <span class="o">=</span> <span class="kp">nil</span><span class="p">)</span>
    <span class="n">sql</span> <span class="o">+=</span> <span class="s2">"; </span><span class="si">#{</span><span class="n">args</span><span class="p">.</span><span class="nf">inspect</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">args</span> <span class="c1"># include bound variables in the output</span>

    <span class="n">activerecord_log</span><span class="p">(</span><span class="n">sql</span><span class="p">)</span> <span class="p">{</span> <span class="k">super</span> <span class="p">}</span>
  <span class="k">end</span>

  <span class="kp">private</span>

  <span class="k">def</span> <span class="nf">activerecord_log</span><span class="p">(</span><span class="n">sql</span><span class="p">)</span>
    <span class="no">ActiveSupport</span><span class="o">::</span><span class="no">Notifications</span><span class="p">.</span><span class="nf">instrument</span><span class="p">(</span>
      <span class="s2">"sql.active_record"</span><span class="p">,</span>
      <span class="ss">sql:        </span><span class="n">sql</span><span class="p">,</span>
      <span class="ss">name:       </span><span class="s2">"Sequel"</span><span class="p">,</span>
      <span class="ss">connection: </span><span class="n">activerecord_connection</span><span class="p">,</span>
      <span class="o">&amp;</span><span class="n">block</span>
    <span class="p">)</span>
  <span class="k">end</span>
  <span class="c1"># ...</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">logger</span> <span class="o">=</span> <span class="no">Logger</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="vg">$stdout</span><span class="p">)</span>

<span class="no">DB</span><span class="p">[</span><span class="ss">:records</span><span class="p">].</span><span class="nf">where</span><span class="p">(</span><span class="ss">foo: </span><span class="s2">"bar"</span><span class="p">).</span><span class="nf">all</span>
<span class="c1">#&gt;&gt; Sequel (0.7ms) SELECT * "records" WHERE "foo" = 'bar'</span>
</code></pre></div></div>

<h2 id="wrapping-it-up">Wrapping it up</h2>

<p>I committed the initial version into the rodauth-rails gem, but I soon realized that getting Sequel to reuse Active Record’s database connection is not specific to Rodauth, so I extracted it into the <a href="https://github.com/janko/sequel-activerecord_connection">sequel-activerecord_connection</a> gem.</p>

<p>I’m glad I did, because it opened doors that weren’t previously open. People can now try out Sequel alongside Active Record without any performance cost or mental overhead, which can be pretty handy given that <a href="/anything-i-want-with-sequel-and-postgres/">Sequel can do lots of things</a> Active Record can’t.</p>

<p>It <a href="https://github.com/janko/sequel-activerecord_connection/commit/0dc74427548e48eb0574b28ebefd65578b490f57">took</a> <a href="https://github.com/janko/sequel-activerecord_connection/commit/a30953269cbe4bc764b055e33efb2a4acff977e2">several</a> <a href="https://github.com/janko/sequel-activerecord_connection/commit/aadfb6e34d0a14bff2fa6e61b905e1e64621460d">iterations</a> to get the implementation right, but extracting it into its own gem helped me focus on this problem in isolation, and converge on the correct behaviour. The end result is a solution that covers much wider use cases than the original problem.</p>]]></content><author><name>Janko Marohnić</name><email>janko@hey.com</email></author><category term="sequel" /><summary type="html"><![CDATA[When I started developing the Rails integration for Rodauth, one of the first problems I needed to solve was how to make Rodauth work seamlessly with Active Record, given that it uses Sequel for database interaction. I believed these two could coexist together, because Sequel is mostly hidden from the Rodauth user anyway, and all that really matters is that Rodauth’s SQL statements get executed on the database.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://janko.io/images/social.jpg" /><media:content medium="image" url="https://janko.io/images/social.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Anything I Want With Sequel And Postgres</title><link href="https://janko.io/anything-i-want-with-sequel-and-postgres/" rel="alternate" type="text/html" title="Anything I Want With Sequel And Postgres" /><published>2021-03-29T00:00:00+00:00</published><updated>2021-03-29T00:00:00+00:00</updated><id>https://janko.io/anything-i-want-with-sequel-and-postgres</id><content type="html" xml:base="https://janko.io/anything-i-want-with-sequel-and-postgres/"><![CDATA[<p>At work I was tasked to migrate our time-series analytics data from CSV file
dumps that we’ve been feeding into Power BI to a dedicated database. Our Rails
app’s primary database is currently MariaDB, but we wanted to have our
analytics data in a separate database either way, so this was a good
opportunity to use Postgres which we’re most comfortable with anyway.</p>

<p>We’re using Active Record for interaction with our primary database, which
gained support for multiple databases in version 6.0. However, given that we
expected the queries to our analytics database would be fairly complex, and
that we’d probably need to be retrieving large quantities of time-series data
(which could be performance-sensitive), I decided it would be a good
opportunity to use <a href="https://github.com/jeremyevans/sequel">Sequel</a> instead.</p>

<p>Thanks to Sequel’s <a href="http://sequel.jeremyevans.net/rdoc/files/doc/postgresql_rdoc.html">advanced Postgres support</a>, I was able to
utilize many cool Postgres features that helped me implement this task
efficiently. Since not all of these features are common, I wanted to showcase
them in this article, and at the same time demonstrate what Sequel is capable
of. :metal:</p>

<h2 id="table-partitioning">Table partitioning</h2>

<p>I mentioned that our analytics data is time-series, which means that we’re
storing snapshots of our product data for each day. This results in a large
number of new records every day, so in order to keep query performance at
acceptable levels, I’ve decided to try out Postgres’ <a href="https://www.postgresql.org/docs/current/ddl-partitioning.html">table partitioning</a>
feature for the first time.</p>

<p>What this feature does is allow you to split data that you would otherwise have
in a single table into multiple tables (“partitions”) based on certain
conditions. These conditions most commonly specify a <strong>range</strong> or <strong>list</strong> of
column values, though you can also partition based on <strong>hash</strong> values.
Postgres’ query planner then determines which partitions it needs to read from
(or write to) based on the SQL query. This can <strong>drammatically improve
performance</strong> for queries where most partitions have been filtered out during
the query planning phase.</p>

<p>Sequel <a href="http://sequel.jeremyevans.net/rdoc/files/doc/postgresql_rdoc.html#label-Creating+Partitioned+Tables">supports Postgres’ table partitioning</a>
out-of-the-box. In order to create a partitioned table (i.e. a table we can
create partitions of), we need to specify the column(s) we want to partition by
(<code class="language-plaintext highlighter-rouge">:partition_by</code>), as well as the type of partitioning (<code class="language-plaintext highlighter-rouge">:partition_type</code>). In
our app, we wanted to have monthly partitions of product data for each client,
so our schema migration contained the following table definition:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">.</span><span class="nf">create_table</span> <span class="ss">:products</span><span class="p">,</span> <span class="ss">partition_by: </span><span class="p">[</span><span class="ss">:instance_id</span><span class="p">,</span> <span class="ss">:date</span><span class="p">],</span> <span class="ss">partition_type: :range</span> <span class="k">do</span>
  <span class="no">Date</span>    <span class="ss">:date</span><span class="p">,</span>        <span class="ss">null: </span><span class="kp">false</span>
  <span class="no">Integer</span> <span class="ss">:instance_id</span><span class="p">,</span> <span class="ss">null: </span><span class="kp">false</span> <span class="c1"># in our app "instances" are e-shops</span>
  <span class="no">String</span>  <span class="ss">:product_id</span><span class="p">,</span>  <span class="ss">null: </span><span class="kp">false</span>

  <span class="c1"># Postgres requires the columns we're partitioning by to be part of the</span>
  <span class="c1"># primary key, so we create a composite primary key</span>
  <span class="n">primary_key</span> <span class="p">[</span><span class="ss">:date</span><span class="p">,</span> <span class="ss">:instance_id</span><span class="p">,</span> <span class="ss">:product_id</span><span class="p">]</span>

  <span class="n">jsonb</span> <span class="ss">:data</span>         <span class="c1"># general data about the product</span>
  <span class="n">jsonb</span> <span class="ss">:competitors</span>  <span class="c1"># data about this product from other competitors</span>
  <span class="n">jsonb</span> <span class="ss">:statistics</span>   <span class="c1"># sales statistics about the product</span>
  <span class="n">jsonb</span> <span class="ss">:applied_rule</span> <span class="c1"># information about our repricing of the product</span>
<span class="k">end</span>
</code></pre></div></div>

<p>The partitioned table above acts as sort of an abstract table, in the sense
that it won’t contain any data by itself, but instead it allows partitions to
be created from it, which will be the ones holding the data. For example, let’s
create a partition of this table which will hold data for an e-shop with ID of
<code class="language-plaintext highlighter-rouge">10</code> for March 2021:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">.</span><span class="nf">create_table?</span> <span class="ss">:products_10_202103</span><span class="p">,</span> <span class="ss">partition_of: :products</span> <span class="k">do</span>
  <span class="n">from</span> <span class="mi">10</span><span class="p">,</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2021</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
  <span class="n">to</span> <span class="mi">10</span><span class="p">,</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2021</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> <span class="c1"># this end is excluded from the range</span>
<span class="k">end</span>
</code></pre></div></div>

<p>The arguments we pass to <code class="language-plaintext highlighter-rouge">from</code> and <code class="language-plaintext highlighter-rouge">to</code> are the values of columns we’ve
specified in <code class="language-plaintext highlighter-rouge">:partition_by</code> on the partitioned table (we have two arguments
because we specified two columns – <code class="language-plaintext highlighter-rouge">:instance_id</code> and <code class="language-plaintext highlighter-rouge">:date</code>). The name of the
table partition is custom, in this example I’ve just chosen a
<code class="language-plaintext highlighter-rouge">products_&lt;INSTANCE_ID&gt;_YYYYMM</code> naming convention. Given that we’re creating
these partitions on-the-fly (as opposed to in a schema migration), I’ve used
Sequel’s <code class="language-plaintext highlighter-rouge">create_table?</code> to handle the case when the partition already exists,
which generates a <code class="language-plaintext highlighter-rouge">CREATE TABLE IF NOT EXISTS</code> query.</p>

<p>Once we’ve created the partitions and populated them with data, we can just
reference the main table in our queries, and Postgres will know which
partition(s) it should direct the queries to.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># queries partition `products_10_202101`</span>
<span class="no">DB</span><span class="p">[</span><span class="ss">:products</span><span class="p">].</span><span class="nf">where</span><span class="p">(</span><span class="ss">instance_id: </span><span class="mi">10</span><span class="p">,</span> <span class="ss">date: </span><span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2021</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)).</span><span class="nf">to_a</span>

<span class="c1"># queries partitions `products_29_202102` and `products_29_202103`</span>
<span class="no">DB</span><span class="p">[</span><span class="ss">:products</span><span class="p">].</span><span class="nf">where</span><span class="p">(</span><span class="ss">instance_id: </span><span class="mi">29</span><span class="p">,</span> <span class="ss">date: </span><span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2021</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span><span class="o">..</span><span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2021</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">31</span><span class="p">)).</span><span class="nf">to_a</span>

<span class="c1"># creates the record in partition `products_13_202012`</span>
<span class="no">DB</span><span class="p">[</span><span class="ss">:products</span><span class="p">].</span><span class="nf">insert</span><span class="p">(</span><span class="ss">instance_id: </span><span class="mi">13</span><span class="p">,</span> <span class="ss">date: </span><span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2020</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">25</span><span class="p">),</span> <span class="ss">product_id: </span><span class="s2">"abc123"</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span>
</code></pre></div></div>

<h2 id="upserts">Upserts</h2>

<p>We have 4 types of product data, each of which is retrieved, aggregated, and
stored in a separate background job. Previously, each background job was
writing to a separate CSV file, but now they would all be writing to a single
table, either creating new records or updating existing records with new data.</p>

<p>The simplest option which is also concurrency-safe was to use Postgres’ <code class="language-plaintext highlighter-rouge">INSERT
... ON CONFLICT ...</code>, also known as “upsert”. Sequel supports upserts with
all its parameters via <a href="http://sequel.jeremyevans.net/rdoc/files/doc/postgresql_rdoc.html#label-INSERT+ON+CONFLICT+Support"><code class="language-plaintext highlighter-rouge">#insert_conflict</code></a>:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">[</span><span class="ss">:products</span><span class="p">]</span>
  <span class="p">.</span><span class="nf">insert_conflict</span> <span class="c1"># by default ignores insert that fails unique constraint violation</span>
  <span class="p">.</span><span class="nf">insert</span><span class="p">(</span><span class="ss">instance_id: </span><span class="mi">10</span><span class="p">,</span> <span class="ss">date: </span><span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2021</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="ss">product_id: </span><span class="s2">"abc123"</span><span class="p">)</span>

<span class="c1"># INSERT INTO products (instance_id, date, product_id)</span>
<span class="c1"># VALUES (10, '2021-01-01', 'abc123')</span>
<span class="c1"># ON CONFLICT DO NOTHING</span>
</code></pre></div></div>

<p>In my task, I needed each background job to only store data it is responsible
for, and that these jobs can be executed in any order. So, the background job
which was responsible for storing general product data into the analytics
database had the following code:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">product_data</span> <span class="c1">#=&gt;</span>
<span class="c1"># [</span>
<span class="c1">#   { instance_id: 10, date: Date.new(2021, 1, 1), product_id: "111", data: { ... } },</span>
<span class="c1">#   { instance_id: 10, date: Date.new(2021, 1, 1), product_id: "222", data: { ... } },</span>
<span class="c1">#   { instance_id: 10, date: Date.new(2021, 1, 1), product_id: "333", data: { ... } },</span>
<span class="c1">#   ...</span>
<span class="c1"># ]</span>

<span class="n">product_data</span><span class="p">.</span><span class="nf">each_slice</span><span class="p">(</span><span class="mi">1000</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">values</span><span class="o">|</span>
  <span class="no">DB</span><span class="p">[</span><span class="ss">:products</span><span class="p">]</span>
    <span class="p">.</span><span class="nf">insert_conflict</span><span class="p">(</span>
      <span class="ss">target: </span><span class="p">[</span><span class="ss">:date</span><span class="p">,</span> <span class="ss">:instance_id</span><span class="p">,</span> <span class="ss">:product_id</span><span class="p">],</span>
      <span class="ss">update: </span><span class="p">{</span> <span class="ss">data: </span><span class="no">Sequel</span><span class="p">[</span><span class="ss">:excluded</span><span class="p">][</span><span class="ss">:data</span><span class="p">]</span> <span class="p">}</span>
    <span class="p">)</span>
    <span class="p">.</span><span class="nf">multi_insert</span><span class="p">(</span><span class="n">values</span><span class="p">)</span>
<span class="k">end</span>

<span class="c1"># INSERT INTO products (...) VALUES (...)</span>
<span class="c1"># ON CONFLICT (date, instance_id, product_id) DO UPDATE data = excluded.data</span>
</code></pre></div></div>

<p>The above inserts values in batches of 1,000 records, and when the record
already exists, only the <code class="language-plaintext highlighter-rouge">data</code> column value is replaced. In general, when a
conflict happens, Postgres exposes the values we’ve tried to insert under the
<code class="language-plaintext highlighter-rouge">excluded</code> qualifier. So, in the <code class="language-plaintext highlighter-rouge">DO UPDATE</code> clause we were able to do <code class="language-plaintext highlighter-rouge">data =
excluded.data</code>, which updates only the <code class="language-plaintext highlighter-rouge">data</code> column. In this case, Postgres
also requires us to specify the column(s) involved in the unique index, which
in our case are <code class="language-plaintext highlighter-rouge">date</code>, <code class="language-plaintext highlighter-rouge">instance_id</code>, and <code class="language-plaintext highlighter-rouge">product_id</code> that form the primary
key.</p>

<h2 id="copy">COPY</h2>

<p>Now that we’ve covered the important bits involved in modifying the code to
write new data into Postgres, what remains is efficiently migrating all the
historical data from our CSV files into Postgres.</p>

<p>The fastest way to import CSV data into a Postgres table is using <code class="language-plaintext highlighter-rouge">COPY FROM</code>,
which Sequel supports via <a href="http://sequel.jeremyevans.net/rdoc-adapters/classes/Sequel/Postgres/Database.html#method-i-copy_into"><code class="language-plaintext highlighter-rouge">#copy_into</code></a>:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">.</span><span class="nf">copy_into</span> <span class="ss">:records</span><span class="p">,</span>
  <span class="ss">format: </span><span class="s2">"csv"</span><span class="p">,</span>
  <span class="ss">options: </span><span class="s2">"HEADERS true"</span><span class="p">,</span>
  <span class="ss">data: </span><span class="no">File</span><span class="p">.</span><span class="nf">foreach</span><span class="p">(</span><span class="s2">"records.csv"</span><span class="p">)</span>
</code></pre></div></div>

<p>In my case, I couldn’t import the CSV files directly into the <code class="language-plaintext highlighter-rouge">products</code>
table, because I wanted to write most of the fields into JSONB columns. So I
first imported the CSV data into a temporary table whose columns matched the
CSV data, and then copied the data from that table into the end <code class="language-plaintext highlighter-rouge">products</code>
table in the desired format.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">data</span> <span class="o">=</span> <span class="no">File</span><span class="p">.</span><span class="nf">foreach</span><span class="p">(</span><span class="s2">"products_10.csv"</span><span class="p">)</span>
<span class="n">columns</span> <span class="o">=</span> <span class="no">File</span><span class="p">.</span><span class="nf">foreach</span><span class="p">(</span><span class="s2">"products_10.csv"</span><span class="p">).</span><span class="nf">first</span><span class="p">.</span><span class="nf">chomp</span><span class="p">.</span><span class="nf">split</span><span class="p">(</span><span class="s2">","</span><span class="p">)</span>
<span class="n">temp_table</span> <span class="o">=</span> <span class="ss">:"products_</span><span class="si">#{</span><span class="no">SecureRandom</span><span class="p">.</span><span class="nf">hex</span><span class="si">}</span><span class="ss">"</span>

<span class="no">DB</span><span class="p">.</span><span class="nf">create_table</span> <span class="n">temp_table</span> <span class="k">do</span>
  <span class="n">columns</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">column</span><span class="o">|</span>
    <span class="no">String</span> <span class="n">column</span><span class="p">.</span><span class="nf">to_sym</span>
  <span class="k">end</span>
<span class="k">end</span>

<span class="no">DB</span><span class="p">.</span><span class="nf">copy_into</span> <span class="n">temp_table</span><span class="p">,</span> <span class="ss">format: </span><span class="s2">"csv"</span><span class="p">,</span> <span class="ss">options: </span><span class="s2">"HEADERS true"</span><span class="p">,</span> <span class="ss">data: </span><span class="n">data</span>

<span class="no">DB</span><span class="p">[</span><span class="n">temp_table</span><span class="p">].</span><span class="nf">paged_each</span><span class="p">.</span><span class="nf">each_slice</span><span class="p">(</span><span class="mi">1000</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">products</span><span class="o">|</span>
  <span class="no">DB</span><span class="p">[</span><span class="ss">:products</span><span class="p">].</span><span class="nf">insert</span> <span class="n">products</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">product</span><span class="o">|</span> <span class="o">...</span> <span class="p">}</span> <span class="c1"># transform into desired format</span>
<span class="k">end</span>

<span class="no">DB</span><span class="p">.</span><span class="nf">drop_table</span> <span class="n">temp_table</span>
</code></pre></div></div>

<h2 id="inserting-from-select">Inserting from SELECT</h2>

<p>Notice how in the last example we were fetching data from the temporary
table, transforming it in Ruby, then writing the result in batches into the
destination table. This is a common way people copy data, but it’s actually
pretty inefficient, both in terms of memory usage and speed.</p>

<p>What we can do instead is transform the data via a <code class="language-plaintext highlighter-rouge">SELECT</code> statement, and then
pass it directly to <code class="language-plaintext highlighter-rouge">INSERT</code>. This way we avoid retrieving any data on the
client side, and we allow Postgres to determine the most efficient way to copy
the data.</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">my_table</span> <span class="p">(</span><span class="n">col1</span><span class="p">,</span> <span class="n">col2</span><span class="p">,</span> <span class="n">col3</span><span class="p">,</span> <span class="p">...)</span>
<span class="k">SELECT</span> <span class="n">val1</span><span class="p">,</span> <span class="n">val2</span><span class="p">,</span> <span class="n">val3</span><span class="p">,</span> <span class="p">...</span> <span class="k">FROM</span> <span class="n">another_table</span> <span class="k">WHERE</span> <span class="p">...</span>
</code></pre></div></div>

<p>Sequel’s <a href="http://sequel.jeremyevans.net/rdoc/classes/Sequel/Dataset.html#method-i-insert"><code class="language-plaintext highlighter-rouge">#insert</code></a> method supports this feature by accepting a
dataset object:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">[</span><span class="ss">:products</span><span class="p">].</span><span class="nf">insert</span> <span class="p">[</span><span class="ss">:instance_id</span><span class="p">,</span> <span class="ss">:date</span><span class="p">,</span> <span class="ss">:product_id</span><span class="p">,</span> <span class="ss">:data</span><span class="p">],</span> <span class="no">DB</span><span class="p">[</span><span class="n">temp_table</span><span class="p">].</span><span class="nf">select</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
</code></pre></div></div>

<p>I’ve covered this topic in more depth in <a href="/inserting-from-datasets-with-sequel/">my recent article</a>, which includes a <a href="/inserting-from-datasets-with-sequel/#measuring-performance">benchmark</a>
illustrating the performance benefits of this approach.</p>

<h2 id="unlogged-tables">Unlogged tables</h2>

<p>Lastly, writing data into a temporary table does create some overhead, which
we can reduce by making the temporary table “unlogged”. With this setting, data
written to this table is not written to Postgres’ write-ahead log (used for
crash recovery), which makes the writing speed considerably faster than in
ordinary tables.</p>

<p>Sequel allows creating unlogged tables by passing the <code class="language-plaintext highlighter-rouge">:unlogged</code> option to
<code class="language-plaintext highlighter-rouge">#create_table</code>:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">.</span><span class="nf">create_table</span> <span class="n">temp_table</span><span class="p">,</span> <span class="ss">unlogged: </span><span class="kp">true</span> <span class="k">do</span>
  <span class="c1"># ...</span>
<span class="k">end</span>
<span class="c1"># CREATE UNLOGGED TABLE products_5ea6fe37d2fde562 (...)</span>
</code></pre></div></div>

<h2 id="loose-count">Loose count</h2>

<p>During this migration, I’ve often wanted to check the total number of records,
to verify that the migration was performed for all of our customers. The
problem is that the regular <code class="language-plaintext highlighter-rouge">SELECT count(*) ...</code> query can be slow for larger
amounts of records.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># can take some time:</span>
<span class="no">DB</span><span class="p">[</span><span class="ss">:products</span><span class="p">].</span><span class="nf">where</span><span class="p">(</span><span class="ss">instance_id: </span><span class="mi">10</span><span class="p">).</span><span class="nf">count</span>
<span class="c1"># SELECT count(*) FROM products WHERE instance_id = 10</span>
</code></pre></div></div>

<p>Luckily, Postgres stores a rough number of records for each table, which can
be retrieved very fast, and in my case that was more than sufficient. I
wouldn’t have found about this Postgres feature if I hadn’t come across
Sequel’s <a href="http://sequel.jeremyevans.net/rdoc-plugins/files/lib/sequel/extensions/pg_loose_count_rb.html">pg_loose_count</a> extension:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">.</span><span class="nf">extension</span> <span class="ss">:pg_loose_count</span>
<span class="no">DB</span><span class="p">.</span><span class="nf">tables</span>
  <span class="p">.</span><span class="nf">grep</span><span class="p">(</span><span class="sr">/products_10_.+/</span><span class="p">)</span> <span class="c1"># select only partitions for e-shop with ID of 10</span>
  <span class="p">.</span><span class="nf">sum</span> <span class="p">{</span> <span class="o">|</span><span class="n">partition</span><span class="o">|</span> <span class="no">DB</span><span class="p">.</span><span class="nf">loose_count</span><span class="p">(</span><span class="n">partition</span><span class="p">)</span> <span class="p">}</span> <span class="c1"># fast count</span>
</code></pre></div></div>

<h2 id="conclusion">Conclusion</h2>

<p>With Sequel and Postgres I was able to use table partitioning to store
time-series data in a way that’s efficient to query, import large amounts of
historical data from CSV files into a temporary unlogged table, and transform
it and write it into the destination table all in SQL, while checking the data
migration progress with Postgres’ loose record counts.</p>

<p>All these Postgres features helped me to efficiently handle time-series data
and import historical data, and I didn’t have to make any comporomises, thanks
to Sequel supporting me every step of the way.</p>]]></content><author><name>Janko Marohnić</name><email>janko@hey.com</email></author><category term="sequel" /><summary type="html"><![CDATA[At work I was tasked to migrate our time-series analytics data from CSV file dumps that we’ve been feeding into Power BI to a dedicated database. Our Rails app’s primary database is currently MariaDB, but we wanted to have our analytics data in a separate database either way, so this was a good opportunity to use Postgres which we’re most comfortable with anyway.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://janko.io/images/social.jpg" /><media:content medium="image" url="https://janko.io/images/social.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Interesting throw/catch behaviour in Ruby</title><link href="https://janko.io/interesting-throw-catch-behaviour-in-ruby/" rel="alternate" type="text/html" title="Interesting throw/catch behaviour in Ruby" /><published>2021-01-24T00:00:00+00:00</published><updated>2021-01-24T00:00:00+00:00</updated><id>https://janko.io/interesting-throw-catch-behaviour-in-ruby</id><content type="html" xml:base="https://janko.io/interesting-throw-catch-behaviour-in-ruby/"><![CDATA[<p>When I was working on integrating <a href="https://github.com/jeremyevans/rodauth">Rodauth</a> with OmniAuth authentication, I
noticed an error warning after upgrading to Rails 6.1, when Rodauth was
redirecting inside a Rails controller action:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">RodauthController</span> <span class="o">&lt;</span> <span class="no">ApplicationController</span>
  <span class="k">def</span> <span class="nf">omniauth</span>
    <span class="c1"># ...</span>
    <span class="n">rodauth</span><span class="p">.</span><span class="nf">login</span><span class="p">(</span><span class="s2">"omniauth"</span><span class="p">)</span> <span class="c1"># logs the session in and redirects</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Could not log "process_action.action_controller" event.
/path/to/actionpack-6.1.1/lib/action_controller/log_subscriber.rb:26:in `block in process_action': undefined method `first' for nil:NilClass (NoMethodError)
</code></pre></div></div>

<p>Since I want the integration between Rodauth and Rails to be as smooth as
possible, I decided to investigate.</p>

<h2 id="diving-in">Diving in</h2>

<p>Let’s see the <code class="language-plaintext highlighter-rouge">ActionController::LogSubscriber</code> source code where the error
happens:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># lib/action_controller/log_subscriber.rb</span>
<span class="k">module</span> <span class="nn">ActionController</span>
  <span class="k">class</span> <span class="nc">LogSubscriber</span> <span class="o">&lt;</span> <span class="no">ActiveSupport</span><span class="o">::</span><span class="no">LogSubscriber</span>
    <span class="c1"># ...</span>
    <span class="k">def</span> <span class="nf">process_action</span><span class="p">(</span><span class="n">event</span><span class="p">)</span>
      <span class="c1"># ...</span>
        <span class="n">status</span> <span class="o">=</span> <span class="n">payload</span><span class="p">[</span><span class="ss">:status</span><span class="p">]</span>

        <span class="k">if</span> <span class="n">status</span><span class="p">.</span><span class="nf">nil?</span> <span class="o">&amp;&amp;</span> <span class="p">(</span><span class="n">exception_class_name</span> <span class="o">=</span> <span class="n">payload</span><span class="p">[</span><span class="ss">:exception</span><span class="p">].</span><span class="nf">first</span><span class="p">)</span> <span class="c1"># &lt;==== the exception happens here</span>
          <span class="n">status</span> <span class="o">=</span> <span class="no">ActionDispatch</span><span class="o">::</span><span class="no">ExceptionWrapper</span><span class="p">.</span><span class="nf">status_code_for_exception</span><span class="p">(</span><span class="n">exception_class_name</span><span class="p">)</span>
        <span class="k">end</span>
      <span class="c1"># ...</span>
    <span class="k">end</span>
    <span class="c1"># ...</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>We can see that the issue happens because <code class="language-plaintext highlighter-rouge">:exception</code> data is missing from the
instrumentation event payload. Let’s look at
<code class="language-plaintext highlighter-rouge">ActionController::Instrumentation</code> next, which is in charge of instrumenting
controller actions:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># lib/action_controller/metal/instrumentation.rb</span>
<span class="k">class</span> <span class="nc">ActionController</span>
  <span class="k">module</span> <span class="nn">Instrumentation</span>
    <span class="k">def</span> <span class="nf">process_action</span><span class="p">(</span><span class="o">*</span><span class="p">)</span>
      <span class="c1"># ...</span>
      <span class="no">ActiveSupport</span><span class="o">::</span><span class="no">Notifications</span><span class="p">.</span><span class="nf">instrument</span><span class="p">(</span><span class="s2">"process_action.action_controller"</span><span class="p">,</span> <span class="n">raw_payload</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">payload</span><span class="o">|</span>
        <span class="n">result</span> <span class="o">=</span> <span class="k">super</span> <span class="c1"># &lt;=== this calls our controller action</span>
        <span class="n">payload</span><span class="p">[</span><span class="ss">:response</span><span class="p">]</span> <span class="o">=</span> <span class="n">response</span>
        <span class="n">payload</span><span class="p">[</span><span class="ss">:status</span><span class="p">]</span>   <span class="o">=</span> <span class="n">response</span><span class="p">.</span><span class="nf">status</span>
        <span class="c1"># ...</span>
      <span class="k">end</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>We can see that, if our controller action raises an exception, the <code class="language-plaintext highlighter-rouge">:status</code>
data will never be set. This ties to the <code class="language-plaintext highlighter-rouge">status.nil?</code> check we’ve seen in the
<code class="language-plaintext highlighter-rouge">ActionController::LogSubscriber</code>.</p>

<p>The remaining part is to find where <code class="language-plaintext highlighter-rouge">:exception</code> is being set. Knowing that
instrumentation is implemented in Active Support, I quickly found
<code class="language-plaintext highlighter-rouge">ActiveSupport::Notifications::Instrumenter</code>:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># lib/active_support/notifications/instrumenter.rb</span>
<span class="k">module</span> <span class="nn">ActiveSupport</span>
  <span class="k">module</span> <span class="nn">Notifications</span>
    <span class="k">class</span> <span class="nc">Instrumenter</span>
      <span class="c1"># ...</span>
      <span class="k">def</span> <span class="nf">instrument</span><span class="p">(</span><span class="nb">name</span><span class="p">,</span> <span class="n">payload</span> <span class="o">=</span> <span class="p">{})</span>
        <span class="c1"># ...</span>
        <span class="k">begin</span>
          <span class="k">yield</span> <span class="n">payload</span> <span class="k">if</span> <span class="nb">block_given?</span>
        <span class="k">rescue</span> <span class="no">Exception</span> <span class="o">=&gt;</span> <span class="n">e</span>
          <span class="n">payload</span><span class="p">[</span><span class="ss">:exception</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="n">e</span><span class="p">.</span><span class="nf">class</span><span class="p">.</span><span class="nf">name</span><span class="p">,</span> <span class="n">e</span><span class="p">.</span><span class="nf">message</span><span class="p">]</span> <span class="c1"># &lt;==== the exception is set here</span>
          <span class="n">payload</span><span class="p">[</span><span class="ss">:exception_object</span><span class="p">]</span> <span class="o">=</span> <span class="n">e</span>
          <span class="k">raise</span> <span class="n">e</span>
        <span class="k">ensure</span>
          <span class="n">finish_with_state</span> <span class="n">listeners_state</span><span class="p">,</span> <span class="nb">name</span><span class="p">,</span> <span class="n">payload</span>
        <span class="k">end</span>
      <span class="k">end</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<h2 id="the-problem">The problem</h2>

<p>When Rodauth redirects, what is actually doing is throwing <code class="language-plaintext highlighter-rouge">:halt</code> with the
rack response. This is how Roda implements redirection, and it’s common practice
in non-Rails web frameworks (Sinatra and Cuba do it too). In our case, throwing
exits from controller action and is caught by the Roda middleware.</p>

<p>Does throwing act the same way as raising an exception does? Initially it
would appear so:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">begin</span>
  <span class="kp">throw</span> <span class="ss">:halt</span>
<span class="k">rescue</span> <span class="no">Exception</span> <span class="o">=&gt;</span> <span class="n">exception</span>
  <span class="nb">puts</span> <span class="s2">"rescue: </span><span class="si">#{</span><span class="n">exception</span><span class="p">.</span><span class="nf">inspect</span><span class="si">}</span><span class="s2">"</span>
  <span class="k">raise</span>
<span class="k">ensure</span>
  <span class="nb">puts</span> <span class="s2">"ensure"</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>rescue: #&lt;UncaughtThrowError: uncaught throw :halt&gt;
ensure
~&gt; uncaught throw :halt (UncaughtThrowError)
</code></pre></div></div>

<p>This makes sense to me, because uncaught throw <em>is</em> an exception. But then why
wasn’t the <code class="language-plaintext highlighter-rouge">rescue</code> block that was supposed to set the <code class="language-plaintext highlighter-rouge">:exception</code> in the
event payload being executed?</p>

<p>The picture starts getting clearer when we wrap the code with a <code class="language-plaintext highlighter-rouge">catch</code>
block:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kp">catch</span><span class="p">(</span><span class="ss">:halt</span><span class="p">)</span> <span class="k">do</span>
  <span class="k">begin</span>
    <span class="kp">throw</span> <span class="ss">:halt</span>
  <span class="k">rescue</span> <span class="no">Exception</span> <span class="o">=&gt;</span> <span class="n">exception</span>
    <span class="nb">puts</span> <span class="s2">"rescue: </span><span class="si">#{</span><span class="n">exception</span><span class="p">.</span><span class="nf">inspect</span><span class="si">}</span><span class="s2">"</span>
    <span class="k">raise</span>
  <span class="k">ensure</span>
    <span class="nb">puts</span> <span class="s2">"ensure"</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ensure
</code></pre></div></div>

<p>We see that in this case the <code class="language-plaintext highlighter-rouge">rescue</code> block isn’t being executed, and this is
precisely our scenario. This actually makes sense when you think about it,
because a <code class="language-plaintext highlighter-rouge">throw</code> with a matching <code class="language-plaintext highlighter-rouge">catch</code> is not anything erroneous, it’s just
a way to do an early return.</p>

<h2 id="the-solution">The solution</h2>

<p>Now we know where the issue is, which is that Rails just wasn’t correctly
handling a <code class="language-plaintext highlighter-rouge">throw</code>/<code class="language-plaintext highlighter-rouge">catch</code> scenario when processing controller actions. <a href="https://github.com/rails/rails/pull/41223">Fixing
it</a> was the easy part.</p>

<p><code class="language-plaintext highlighter-rouge">throw</code>/<code class="language-plaintext highlighter-rouge">catch</code> is probably something you’ll rarely use, but it does have its
use cases. I hope this article taught you a bit more about this lesser known
Ruby feature.</p>]]></content><author><name>Janko Marohnić</name><email>janko@hey.com</email></author><category term="til" /><summary type="html"><![CDATA[When I was working on integrating Rodauth with OmniAuth authentication, I noticed an error warning after upgrading to Rails 6.1, when Rodauth was redirecting inside a Rails controller action:]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://janko.io/images/social.jpg" /><media:content medium="image" url="https://janko.io/images/social.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Multifactor Authentication in Rails with Rodauth</title><link href="https://janko.io/adding-multifactor-authentication-in-rails-with-rodauth/" rel="alternate" type="text/html" title="Multifactor Authentication in Rails with Rodauth" /><published>2020-12-21T00:00:00+00:00</published><updated>2020-12-21T00:00:00+00:00</updated><id>https://janko.io/adding-multifactor-authentication-in-rails-with-rodauth</id><content type="html" xml:base="https://janko.io/adding-multifactor-authentication-in-rails-with-rodauth/"><![CDATA[<p>Multi-factor authentication or MFA (generalized two-factor authentication or
2FA) is a method of authentication where the user is required to provide two or
more pieces of evidence (“factors”) in order to be granted access. Typically
the user would first prove knowledge of something only they <em>know</em> (e.g. their
password), and then prove posession of something only they <em>own</em> (e.g. another
device). This provides an extra layer of security for the user’s account.</p>

<p>Most common multifactor authentication methods include:</p>

<ul>
  <li>
    <p><strong>TOTP</strong> (Time-based One-Time Passwords) – user has an app installed on
their device that displays the authentication code, which is refreshed every
30 seconds</p>
  </li>
  <li>
    <p><strong>SMS codes</strong> – user receives authentication codes on their phone via SMS
when the application requests them</p>
  </li>
  <li>
    <p><strong>Recovery codes</strong> – user is given a fixed set of one-time codes they can
enter when logging in (this is typically used as a backup method)</p>
  </li>
  <li>
    <p><strong><a href="https://webauthn.io/">WebAuthn</a></strong> – user authenticates themselves using a <a href="https://en.wikipedia.org/wiki/Universal_2nd_Factor">security key</a> or
built-in platform biometric sensors (e.g. fingerprint)</p>
  </li>
</ul>

<p>In this article, I want to show you how to add multifactor authentication to
a Rails app using <a href="https://github.com/jeremyevans/rodauth/">Rodauth</a>, which has built-in support for each of the
multifactor authentication methods mentioned above. Compared to alternatives<sup id="fnref:1"><a href="#fn:1" class="footnote" rel="footnote" role="doc-noteref">1</a></sup>,
Rodauth provides a much more integrated experience by shipping with complete
endpoints, default HTML templates, session management, lockout logic and
more<sup id="fnref:2"><a href="#fn:2" class="footnote" rel="footnote" role="doc-noteref">2</a></sup>. To keep the tutorial focused, we’ll be implementing just the first
three methods, as they’re by far the most common.</p>

<p>We’ll be using the <a href="https://github.com/janko/rodauth-rails">rodauth-rails</a> gem, and we’ll be continuing off of the
application we started building in <a href="/adding-authentication-in-rails-with-rodauth/">my previous article</a>. The
goal functionality: allow users to set up TOTP as their primary MFA method, and
use SMS codes and recovery codes as backup MFA methods.</p>

<h2 id="totp">TOTP</h2>

<p>The TOTP functionality is provided by Rodauth’s <a href="http://rodauth.jeremyevans.net/rdoc/files/doc/otp_rdoc.html"><code class="language-plaintext highlighter-rouge">otp</code></a> feature. It
depends on the <a href="https://github.com/mdp/rotp">rotp</a> and <a href="https://github.com/whomwah/rqrcode">rqrcode</a> gems, so let’s first install those:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>bundle add rotp rqrcode
</code></pre></div></div>

<p>Next, we need to create the required database table. For this we’ll use the
migration generator provided by rodauth-rails:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate rodauth:migration otp
<span class="c"># create  db/migrate/20201214200106_create_rodauth_otp.rb</span>

<span class="nv">$ </span>rails db:migrate
<span class="c"># == 20201214200106 CreateRodauthOtp: migrating =======================</span>
<span class="c"># -- create_table(:account_otp_keys)</span>
<span class="c"># == 20201214200106 CreateRodauthOtp: migrated ========================</span>
</code></pre></div></div>

<p>Now we can enable the <code class="language-plaintext highlighter-rouge">otp</code> feature in our Rodauth configuration:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># ...</span>
    <span class="n">enable</span> <span class="ss">:otp</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>This adds the following routes to our application:</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">/otp-auth</code> – authenticate via TOTP code</li>
  <li><code class="language-plaintext highlighter-rouge">/otp-setup</code> – set up TOTP authentication</li>
  <li><code class="language-plaintext highlighter-rouge">/otp-disable</code> – disable TOTP authentication</li>
  <li><code class="language-plaintext highlighter-rouge">/multifactor-manage</code> – set up or disable available MFA methods</li>
  <li><code class="language-plaintext highlighter-rouge">/multifactor-auth</code> – authenticate via available MFA methods</li>
  <li><code class="language-plaintext highlighter-rouge">/multifactor-disable</code> – disable all MFA methods</li>
</ul>

<p>To allow the user to configure MFA, let’s display a link to the
<code class="language-plaintext highlighter-rouge">/multifactor-manage</code> route for managing MFA methods in our views:</p>

<div class="language-erb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- app/views/application/_navbar.html.erb --&gt;</span>
<span class="cp">&lt;%</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">logged_in?</span> <span class="cp">%&gt;</span>
  <span class="c">&lt;!-- ... ---&gt;</span>
  <span class="cp">&lt;%=</span> <span class="n">link_to</span> <span class="s2">"Manage MFA"</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">two_factor_manage_path</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"dropdown-item"</span> <span class="cp">%&gt;</span>
  <span class="c">&lt;!-- ... ---&gt;</span>
<span class="cp">&lt;%</span> <span class="k">end</span> <span class="cp">%&gt;</span>
</code></pre></div></div>

<p>Now when the user logs in and clicks on “Manage MFA”, they’ll get redirected to
the OTP setup page that Rodauth provides out-of-the-box<sup id="fnref:3"><a href="#fn:3" class="footnote" rel="footnote" role="doc-noteref">3</a></sup>:</p>

<p><img src="/images/rodauth-otp-setup.png" alt="Rodauth OTP setup page" /></p>

<p>The user can now scan the QR code using an authenticator app such as Google
Authenticator, Microsoft Authenticator or Authy, and enter the OTP code (along
with their current password) to finish setting up OTP. As a developer, you can
generate the code from the OTP secret using the ROTP gem:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rotp <span class="nt">--secret</span> omo2p3movepqyc222rp54v3cic7ky2au
409761
</code></pre></div></div>

<p>When the user with OTP set up logs in the next time, we want them to be
automatically redirected to the OTP auth page. We can achieve this by requiring
logged in users that have MFA set up to authenticate with 2nd factor, and
tweaking the flash messages to make it feel like part of one signin:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_app.rb</span>
<span class="k">class</span> <span class="nc">RodauthApp</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">App</span>
  <span class="c1"># ...</span>
  <span class="n">route</span> <span class="k">do</span> <span class="o">|</span><span class="n">r</span><span class="o">|</span>
    <span class="c1"># ...</span>
    <span class="c1"># require MFA if the user is logged in and has MFA setup</span>
    <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">uses_two_factor_authentication?</span>
      <span class="n">rodauth</span><span class="p">.</span><span class="nf">require_two_factor_authenticated</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># ...</span>
    <span class="c1"># don't show error message when redirected after login</span>
    <span class="n">two_factor_need_authentication_error_flash</span> <span class="p">{</span> <span class="n">flash</span><span class="p">[</span><span class="ss">:notice</span><span class="p">]</span> <span class="o">==</span> <span class="n">login_notice_flash</span> <span class="p">?</span> <span class="kp">nil</span> <span class="p">:</span> <span class="k">super</span><span class="p">()</span> <span class="p">}</span>
    <span class="c1"># show generic authentication message</span>
    <span class="n">two_factor_auth_notice_flash</span> <span class="p">{</span> <span class="n">login_notice_flash</span> <span class="p">}</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p><img src="/images/rodauth-otp-auth.png" alt="Rodauth TOTP authentication page" /></p>

<h2 id="recovery-codes">Recovery codes</h2>

<p>After the user sets up TOTP, it’s recommended to also generate a set of
“recovery” codes for them to save somewhere, which they can use on login in
case they lose access to their TOTP device. This functionality is provided by
Rodauth’s <a href="http://rodauth.jeremyevans.net/rdoc/files/doc/recovery_codes_rdoc.html"><code class="language-plaintext highlighter-rouge">recovery_codes</code></a> feature.</p>

<p>Let’s start by creating the required database table:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate rodauth:migration recovery_codes
<span class="c"># create  db/migrate/20201214200106_create_rodauth_recovery_codes.rb</span>

<span class="nv">$ </span>rails db:migrate
<span class="c"># == 20201217071036 CreateRodauthRecoveryCodes: migrating =======================</span>
<span class="c"># -- create_table(:account_recovery_codes, {:primary_key=&gt;[:id, :code]})</span>
<span class="c"># == 20201217071036 CreateRodauthRecoveryCodes: migrated ========================</span>
</code></pre></div></div>

<p>And enabling the <code class="language-plaintext highlighter-rouge">recovery_codes</code> feature in our Rodauth configuration:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># ...</span>
    <span class="n">enable</span> <span class="ss">:otp</span><span class="p">,</span> <span class="ss">:recovery_codes</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>This adds the following routes to our app:</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">/recovery-auth</code> – authenticate via a recovery code</li>
  <li><code class="language-plaintext highlighter-rouge">/recovery-codes</code> – view &amp; add recovery codes</li>
</ul>

<p>We’ll now override the <code class="language-plaintext highlighter-rouge">after_otp_setup</code> hook to display recovery codes to the
user after they’ve successfully set up TOTP, instead of the default redirect.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># ...</span>
    <span class="c1"># automatically generate recovery codes after enabling first MFA method</span>
    <span class="n">auto_add_recovery_codes?</span> <span class="kp">true</span>
    <span class="c1"># automatically remove recovery codes after disabling last MFA method</span>
    <span class="n">auto_remove_recovery_codes?</span> <span class="kp">true</span>
    <span class="c1"># display recovery codes after TOTP setup</span>
    <span class="n">after_otp_setup</span> <span class="k">do</span>
      <span class="n">set_notice_now_flash</span> <span class="s2">"</span><span class="si">#{</span><span class="n">otp_setup_notice_flash</span><span class="si">}</span><span class="s2">, please make note of your recovery codes"</span>
      <span class="n">return_response</span> <span class="n">add_recovery_codes_view</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>We’ll also override the default Rodauth template to display the recovery codes
in a nicer way. For convenience, we’ll add a download link for the recovery
codes as well. Instead of adding a new endpoint, which would have to be
password-protected to maintain security, we’ll just implement the download link
in plain HTML using a data URL and <code class="language-plaintext highlighter-rouge">download</code> attribute.</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate rodauth:views recovery_codes
</code></pre></div></div>
<div class="language-erb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- app/views/rodauth/add_recovery_codes.html.erb --&gt;</span>
<span class="cp">&lt;%</span> <span class="n">content_for</span> <span class="ss">:title</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">add_recovery_codes_page_title</span> <span class="cp">%&gt;</span>

<span class="cp">&lt;%</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">recovery_codes</span><span class="p">.</span><span class="nf">any?</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;p</span> <span class="na">class=</span><span class="s">"my-3"</span><span class="nt">&gt;</span>
    Copy these recovery codes to a safe location.
    You can also download them <span class="cp">&lt;%=</span> <span class="n">link_to</span> <span class="s2">"here"</span><span class="p">,</span> <span class="s2">"data:,</span><span class="si">#{</span><span class="n">rodauth</span><span class="p">.</span><span class="nf">recovery_codes</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span> <span class="ss">download: </span><span class="s2">"myapp-recovery-codes.txt"</span> <span class="cp">%&gt;</span>.
  <span class="nt">&lt;/p&gt;</span>

  <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"d-inline-block mb-3 border border-info rounded px-3 py-2"</span><span class="nt">&gt;</span>
    <span class="cp">&lt;%</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">recovery_codes</span><span class="p">.</span><span class="nf">each_slice</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">code1</span><span class="p">,</span> <span class="n">code2</span><span class="o">|</span> <span class="cp">%&gt;</span>
      <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"row text-info text-left"</span><span class="nt">&gt;</span>
        <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"col-lg my-1 font-monospace"</span><span class="nt">&gt;</span><span class="cp">&lt;%=</span> <span class="n">code1</span> <span class="cp">%&gt;</span><span class="nt">&lt;/div&gt;</span>
        <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"col-lg my-1 font-monospace"</span><span class="nt">&gt;</span><span class="cp">&lt;%=</span> <span class="n">code2</span> <span class="cp">%&gt;</span><span class="nt">&lt;/div&gt;</span>
      <span class="nt">&lt;/div&gt;</span>
    <span class="cp">&lt;%</span> <span class="k">end</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;/div&gt;</span>
<span class="cp">&lt;%</span> <span class="k">end</span> <span class="cp">%&gt;</span>

<span class="c">&lt;!-- Used for filling in missing recovery codes later on --&gt;</span>
<span class="cp">&lt;%</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">can_add_recovery_codes?</span> <span class="cp">%&gt;</span>
  <span class="cp">&lt;%=</span><span class="o">=</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">add_recovery_codes_heading</span> <span class="cp">%&gt;</span>
  <span class="cp">&lt;%=</span> <span class="n">render</span> <span class="ss">template: </span><span class="s2">"rodauth/recovery_codes"</span><span class="p">,</span> <span class="ss">layout: </span><span class="kp">false</span> <span class="cp">%&gt;</span>
<span class="cp">&lt;%</span> <span class="k">end</span> <span class="cp">%&gt;</span>
</code></pre></div></div>

<p>When the user now sets up TOTP, they will be shown a page like this:</p>

<p><img src="/images/rodauth-recovery-view.png" alt="Rodauth page for viewing and downloading recovery codes" /></p>

<p>And when they log into their account the next time, on the multifactor auth
page they can choose to enter a recovery code instead of TOTP.</p>

<p><img src="/images/rodauth-recovery-auth.png" alt="Multifactor auth page with OTP and recovery codes options" /></p>

<h2 id="sms-codes">SMS codes</h2>

<p>In addition to TOTP, it’s good practice to also provide the ability to use SMS
codes for 2nd factor authentication. Rodauth provides a specialized
<a href="http://rodauth.jeremyevans.net/rdoc/files/doc/sms_codes_rdoc.html"><code class="language-plaintext highlighter-rouge">sms_codes</code></a> feature for this.</p>

<p>To set it up, we again create the required database table:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate rodauth:migration sms_codes
<span class="c"># create  db/migrate/20201219173710_create_rodauth_sms_codes.rb</span>

<span class="nv">$ </span>rails db:migrate
<span class="c"># == 20201219173710 CreateRodauthSmsCodes: migrating ==================</span>
<span class="c"># -- create_table(:account_sms_codes)</span>
<span class="c"># == 20201219173710 CreateRodauthSmsCodes: migrated ===================</span>
</code></pre></div></div>

<p>And enable the <code class="language-plaintext highlighter-rouge">sms_codes</code> feature in the Rodauth configuration:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># ...</span>
    <span class="n">enable</span> <span class="ss">:otp</span><span class="p">,</span> <span class="ss">:recovery_codes</span><span class="p">,</span> <span class="ss">:sms_codes</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>This adds the following routes to our app:</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">/sms-request</code> – request the SMS code to be sent</li>
  <li><code class="language-plaintext highlighter-rouge">/sms-auth</code> – authenticate via an SMS code</li>
  <li><code class="language-plaintext highlighter-rouge">/sms-setup</code> – set up SMS codes authentication</li>
  <li><code class="language-plaintext highlighter-rouge">/sms-confirm</code> – confirm the provided phone number</li>
  <li><code class="language-plaintext highlighter-rouge">/sms-disable</code> – disable SMS codes authentication</li>
</ul>

<p>When an SMS code is requested, Rodauth calls the <code class="language-plaintext highlighter-rouge">sms_send</code> method with the
configured phone number and a corresponding text message. This method isn’t
defined by default, since Rodauth doesn’t know how we want to send the SMS,
instead we’re expected to implement <code class="language-plaintext highlighter-rouge">sms_send</code>:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># ...</span>
    <span class="n">sms_send</span> <span class="k">do</span> <span class="o">|</span><span class="n">phone</span><span class="p">,</span> <span class="n">message</span><span class="o">|</span>
      <span class="c1"># we need to implement this</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>We’ll use <a href="https://www.twilio.com/">Twilio</a> for sending SMS messages. Assuming we’ve set up an account,
we’ll add the account SID, auth token, and phone number to Rails credentials:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails credentials:edit
</code></pre></div></div>
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">twilio</span><span class="pi">:</span>
  <span class="na">account_sid</span><span class="pi">:</span> <span class="s">&lt;YOUR_ACCOUNT_SID&gt;</span>
  <span class="na">auth_token</span><span class="pi">:</span> <span class="s">&lt;YOUR_AUTH_TOKEN&gt;</span>
  <span class="na">phone_number</span><span class="pi">:</span> <span class="s">&lt;YOUR_PHONE_NUMBER&gt;</span>
</code></pre></div></div>

<p>Next, we’ll install the <a href="https://github.com/twilio/twilio-ruby">twilio-ruby</a> gem, and create a wrapper class for the
Twilio client that uses the configured credentials:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>bundle add twilio-ruby
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/twilio_client.rb</span>
<span class="k">class</span> <span class="nc">TwilioClient</span>
  <span class="no">Error</span>              <span class="o">=</span> <span class="no">Class</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">StandardError</span><span class="p">)</span>
  <span class="no">InvalidPhoneNumber</span> <span class="o">=</span> <span class="no">Class</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">Error</span><span class="p">)</span>

  <span class="k">def</span> <span class="nf">initialize</span>
    <span class="vi">@account_sid</span> <span class="o">=</span> <span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">credentials</span><span class="p">.</span><span class="nf">twilio</span><span class="p">.</span><span class="nf">account_sid!</span>
    <span class="vi">@auth_token</span> <span class="o">=</span> <span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">credentials</span><span class="p">.</span><span class="nf">twilio</span><span class="p">.</span><span class="nf">auth_token!</span>
    <span class="vi">@phone_number</span> <span class="o">=</span> <span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">credentials</span><span class="p">.</span><span class="nf">twilio</span><span class="p">.</span><span class="nf">phone_number!</span>
  <span class="k">end</span>

  <span class="k">def</span> <span class="nf">send_sms</span><span class="p">(</span><span class="n">to</span><span class="p">,</span> <span class="n">message</span><span class="p">)</span>
    <span class="n">client</span><span class="p">.</span><span class="nf">messages</span><span class="p">.</span><span class="nf">create</span><span class="p">(</span><span class="ss">from: </span><span class="vi">@phone_number</span><span class="p">,</span> <span class="ss">to: </span><span class="n">to</span><span class="p">,</span> <span class="ss">body: </span><span class="n">message</span><span class="p">)</span>
  <span class="k">rescue</span> <span class="no">Twilio</span><span class="o">::</span><span class="no">REST</span><span class="o">::</span><span class="no">RestError</span> <span class="o">=&gt;</span> <span class="n">error</span>
    <span class="c1"># more details here: https://www.twilio.com/docs/api/errors/21211</span>
    <span class="k">raise</span> <span class="no">TwilioClient</span><span class="o">::</span><span class="no">InvalidPhoneNumber</span><span class="p">,</span> <span class="n">error</span><span class="p">.</span><span class="nf">message</span> <span class="k">if</span> <span class="n">error</span><span class="p">.</span><span class="nf">code</span> <span class="o">==</span> <span class="mi">21211</span>
    <span class="k">raise</span> <span class="no">TwilioClient</span><span class="o">::</span><span class="no">Error</span><span class="p">,</span> <span class="n">error</span><span class="p">.</span><span class="nf">message</span>
  <span class="k">end</span>

  <span class="k">def</span> <span class="nf">client</span>
    <span class="no">Twilio</span><span class="o">::</span><span class="no">REST</span><span class="o">::</span><span class="no">Client</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="vi">@account_sid</span><span class="p">,</span> <span class="vi">@auth_token</span><span class="p">)</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Finally, we’ll implement <code class="language-plaintext highlighter-rouge">sms_send</code> using our new <code class="language-plaintext highlighter-rouge">TwilioClient</code> class. We’ll
convert SMS sending errors into validation errors, making sure we roll back
the wrapping database transaction to prevent phone number &amp; code from being
persisted:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># ...</span>
    <span class="n">sms_send</span> <span class="k">do</span> <span class="o">|</span><span class="n">phone</span><span class="p">,</span> <span class="n">message</span><span class="o">|</span>
      <span class="n">twilio</span> <span class="o">=</span> <span class="no">TwilioClient</span><span class="p">.</span><span class="nf">new</span>
      <span class="n">twilio</span><span class="p">.</span><span class="nf">send_sms</span><span class="p">(</span><span class="n">phone</span><span class="p">,</span> <span class="n">message</span><span class="p">)</span>
    <span class="k">rescue</span> <span class="no">TwilioClient</span><span class="o">::</span><span class="no">Error</span> <span class="o">=&gt;</span> <span class="n">error</span>
      <span class="n">db</span><span class="p">.</span><span class="nf">rollback_on_exit</span>
      <span class="n">throw_error_status</span><span class="p">(</span><span class="mi">422</span><span class="p">,</span> <span class="n">sms_phone_param</span><span class="p">,</span> <span class="n">sms_invalid_phone_message</span><span class="p">)</span> <span class="k">if</span> <span class="n">error</span><span class="p">.</span><span class="nf">is_a?</span><span class="p">(</span><span class="no">TwilioClient</span><span class="o">::</span><span class="no">InvalidPhoneNumber</span><span class="p">)</span>
      <span class="n">throw_error_status</span><span class="p">(</span><span class="mi">500</span><span class="p">,</span> <span class="n">sms_phone_param</span><span class="p">,</span> <span class="s2">"sending the SMS code failed"</span><span class="p">)</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>When the user now visits the SMS authentication setup page on the multifactor
manage page, they can enter their phone number and password, and then enter the
SMS code they received to finish the SMS authentication setup.</p>

<p><img src="/images/rodauth-sms-setup.png" alt="Rodauth SMS authentication setup page" /></p>

<p>Afterwards, when the user logs in the next time, in addition to authenticating
via TOTP or a recovery code, they’ll now also be able to choose to authenticate
via SMS.</p>

<h2 id="disabling-multifactor-authentication">Disabling multifactor authentication</h2>

<p>In addition to setup and authentication, Rodauth also provides endpoints for
disabling any MFA method, which require the user to confirm their password:</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">/otp-disable</code> – disable OTP authentication</li>
  <li><code class="language-plaintext highlighter-rouge">/sms-disable</code> – disable multifactor authentication</li>
  <li><code class="language-plaintext highlighter-rouge">/multifactor-disable</code> – disable all multifactor methods</li>
</ul>

<p>The links for disabling MFA methods that have previously been set up are
automatically displayed on the multifactor manage page:</p>

<p><img src="/images/rodauth-mfa-disable.png" alt="Rodauth links for disabling configured MFA methods" /></p>

<p>Disabling a MFA method will take care of deleting any records associated to
that account from the corresponding database table.</p>

<h2 id="closing-words">Closing words</h2>

<p>In this tutorial we’ve shown how to add multifactor authentication
functionality in Rails with Rodauth and rodauth-rails. We’ve enabled the
user to set up TOTP as their primary MFA method, after which they receive a set
of recovery codes, and have the possibility to also set up SMS as a backup MFA
method.</p>

<p>We’ve seen that Rodauth ships with complete endpoints and default HTML
templates for managing multiple MFA methods, and generally provides a much more
integrated experience compared to the alternatives. Given that multifactor
authentication is becoming an increasingly common requirement, it’s very useful
to have a framework that supports it with the same level of standard as the
other authentication features.</p>

<div class="footnotes" role="doc-endnotes">
  <ol>
    <li id="fn:1">
      <p>At the time of writing, most popular alternatives are <a href="https://github.com/tinfoil/devise-two-factor">devise-two-factor</a>, <a href="https://github.com/heapsource/active_model_otp">active_model_otp</a>, and <a href="https://github.com/Houdini/two_factor_authentication">two_factor_authentication</a>. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:2">
      <p>See the source code for <a href="https://github.com/jeremyevans/rodauth/blob/master/lib/rodauth/features/otp.rb">OTP</a>, <a href="https://github.com/jeremyevans/rodauth/blob/master/lib/rodauth/features/sms_codes.rb">SMS Codes</a>, <a href="https://github.com/jeremyevans/rodauth/blob/master/lib/rodauth/features/recovery_codes.rb">Recovery Codes</a>, and <a href="https://github.com/jeremyevans/rodauth/blob/master/lib/rodauth/features/two_factor_base.rb">Two Factor Base</a> for more details. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:3">
      <p>You can override the default template by running <code class="language-plaintext highlighter-rouge">rails generate rodauth:views otp</code> and modifying <code class="language-plaintext highlighter-rouge">app/views/rodauth/otp_setup.html.erb</code>. <a href="#fnref:3" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
  </ol>
</div>]]></content><author><name>Janko Marohnić</name><email>janko@hey.com</email></author><category term="rodauth" /><summary type="html"><![CDATA[Multi-factor authentication or MFA (generalized two-factor authentication or 2FA) is a method of authentication where the user is required to provide two or more pieces of evidence (“factors”) in order to be granted access. Typically the user would first prove knowledge of something only they know (e.g. their password), and then prove posession of something only they own (e.g. another device). This provides an extra layer of security for the user’s account.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://janko.io/images/social.jpg" /><media:content medium="image" url="https://janko.io/images/social.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Rails Authentication with Rodauth</title><link href="https://janko.io/adding-authentication-in-rails-with-rodauth/" rel="alternate" type="text/html" title="Rails Authentication with Rodauth" /><published>2020-11-19T00:00:00+00:00</published><updated>2020-11-19T00:00:00+00:00</updated><id>https://janko.io/adding-authentication-in-rails-with-rodauth</id><content type="html" xml:base="https://janko.io/adding-authentication-in-rails-with-rodauth/"><![CDATA[<p>In this tutorial, we’ll show how to add fully functional authentication and
account management functionality into a Rails app, using the <strong><a href="https://github.com/jeremyevans/rodauth">Rodauth</a></strong>
authentication framework. Rodauth has many advantages over the mainstream
alternatives such as Devise, Sorcery, Clearance, and Authlogic, see my
<a href="https://janko.io/rodauth-a-refreshing-authentication-solution-for-ruby/">previous article</a> for an introduction.</p>

<p>We’ll be working with a fresh Rails app using PostgresSQL, Hotwire,
<a href="https://getbootstrap.com/">Bootstrap</a>, home page, navbar, flash messages, and posts scaffold setup.</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails new blog <span class="nt">--database</span><span class="o">=</span>postgresql <span class="nt">--css</span><span class="o">=</span>bootstrap
<span class="nv">$ </span><span class="nb">cd </span>blog
<span class="nv">$ </span>rails db:create
<span class="nv">$ </span>rails generate controller home index
<span class="nv">$ </span>rails generate scaffold post title:string body:text
<span class="nv">$ </span>rails db:migrate
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># config/routes.rb</span>
<span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">routes</span><span class="p">.</span><span class="nf">draw</span> <span class="k">do</span>
  <span class="n">root</span> <span class="ss">to: </span><span class="s2">"home#index"</span>
  <span class="n">resources</span> <span class="ss">:posts</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-erb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- app/views/layouts/application.html.erb --&gt;</span>
<span class="c">&lt;!-- ... --&gt;</span>
  <span class="nt">&lt;body&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">render</span> <span class="s2">"navbar"</span> <span class="cp">%&gt;</span>

    <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"container"</span><span class="nt">&gt;</span>
      <span class="cp">&lt;%=</span> <span class="n">render</span> <span class="s2">"flash"</span> <span class="cp">%&gt;</span>
      <span class="cp">&lt;%=</span> <span class="k">yield</span> <span class="cp">%&gt;</span>
    <span class="nt">&lt;/div&gt;</span>
  <span class="nt">&lt;/body&gt;</span>
<span class="c">&lt;!-- ... --&gt;</span>
</code></pre></div></div>
<div class="language-erb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- app/views/application/_flash.html.erb --&gt;</span>
<span class="cp">&lt;%</span> <span class="k">if</span> <span class="n">notice</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"alert alert-success"</span><span class="nt">&gt;</span><span class="cp">&lt;%=</span> <span class="n">notice</span> <span class="cp">%&gt;</span><span class="nt">&lt;/div&gt;</span>
<span class="cp">&lt;%</span> <span class="k">end</span> <span class="cp">%&gt;</span>
<span class="cp">&lt;%</span> <span class="k">if</span> <span class="n">alert</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"alert alert-danger"</span><span class="nt">&gt;</span><span class="cp">&lt;%=</span> <span class="n">alert</span> <span class="cp">%&gt;</span><span class="nt">&lt;/div&gt;</span>
<span class="cp">&lt;%</span> <span class="k">end</span> <span class="cp">%&gt;</span>
</code></pre></div></div>
<div class="language-erb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- app/views/application/_navbar.html.erb --&gt;</span>
<span class="nt">&lt;nav</span> <span class="na">class=</span><span class="s">"navbar navbar-expand-sm navbar-light bg-light border-bottom mb-4"</span><span class="nt">&gt;</span>
  <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"container"</span><span class="nt">&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">link_to</span> <span class="s2">"Rails App"</span><span class="p">,</span> <span class="n">root_path</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"navbar-brand"</span> <span class="cp">%&gt;</span>

    <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"navbar-collapse"</span><span class="nt">&gt;</span>
      <span class="nt">&lt;ul</span> <span class="na">class=</span><span class="s">"navbar-nav"</span><span class="nt">&gt;</span>
        <span class="nt">&lt;li</span> <span class="na">class=</span><span class="s">"nav-item"</span><span class="nt">&gt;</span>
          <span class="cp">&lt;%=</span> <span class="n">link_to</span> <span class="s2">"Posts"</span><span class="p">,</span> <span class="n">posts_path</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"nav-link </span><span class="si">#{</span><span class="s2">"active"</span> <span class="k">if</span> <span class="n">request</span><span class="p">.</span><span class="nf">path</span><span class="p">.</span><span class="nf">start_with?</span><span class="p">(</span><span class="s2">"/posts"</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span> <span class="cp">%&gt;</span>
        <span class="nt">&lt;/li&gt;</span>
      <span class="nt">&lt;/ul&gt;</span>
    <span class="nt">&lt;/div&gt;</span>
  <span class="nt">&lt;/div&gt;</span>
<span class="nt">&lt;/nav&gt;</span>
</code></pre></div></div>

<h2 id="installing-rodauth">Installing Rodauth</h2>

<p>Let’s start by adding the <a href="https://github.com/janko/rodauth-rails">rodauth-rails</a> gem to our Gemfile:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>bundle add rodauth-rails
</code></pre></div></div>

<p>Next, we’ll run the <code class="language-plaintext highlighter-rouge">rodauth:install</code> generator provided by rodauth-rails:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate rodauth:install

<span class="c"># create  db/migrate/20200820215819_create_rodauth.rb</span>
<span class="c"># create  config/initializers/rodauth.rb</span>
<span class="c"># create  config/initializers/sequel.rb</span>
<span class="c"># create  app/misc/rodauth_app.rb</span>
<span class="c"># create  app/misc/rodauth_main.rb</span>
<span class="c"># create  app/controllers/rodauth_controller.rb</span>
<span class="c"># create  app/models/account.rb</span>
<span class="c"># create  app/mailers/rodauth_mailer.rb</span>
</code></pre></div></div>

<p>This will create the Rodauth app and some default Rodauth configuration,
configure <a href="https://github.com/jeremyevans/sequel">Sequel</a> which Rodauth uses for database interaction to <a href="https://github.com/janko/sequel-activerecord_connection">reuse Active
Record’s database connection</a>, and generate a
migration that will create tables for the loaded Rodauth features. Let’s run
the migration:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails db:migrate

<span class="c"># == CreateRodauth: migrating ==========================</span>
<span class="c"># -- create_table(:accounts)</span>
<span class="c"># -- create_table(:account_password_hashes)</span>
<span class="c"># -- create_table(:account_password_reset_keys)</span>
<span class="c"># -- create_table(:account_verification_keys)</span>
<span class="c"># -- create_table(:account_login_change_keys)</span>
<span class="c"># -- create_table(:account_remember_keys)</span>
<span class="c"># == CreateRodauth: migrated ===========================</span>
</code></pre></div></div>

<p>We’ll also need to set Action Mailer’s default URL options for Rodauth to be
able to generate email links in <code class="language-plaintext highlighter-rouge">RodauthMailer</code>:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># config/environments/development.rb</span>
<span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span>
  <span class="c1"># ...</span>
  <span class="n">config</span><span class="p">.</span><span class="nf">action_mailer</span><span class="p">.</span><span class="nf">default_url_options</span> <span class="o">=</span> <span class="p">{</span> <span class="ss">host: </span><span class="s1">'localhost'</span><span class="p">,</span> <span class="ss">port: </span><span class="mi">3000</span> <span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>

<p>After restarting the Rails server, we should be able to open the
<code class="language-plaintext highlighter-rouge">/create-account</code> page and see Rodauth’s default registration form.</p>

<p><img src="/images/rodauth-create-account.png" alt="Rodauth create account page" /></p>

<h2 id="adding-authentication-links">Adding authentication links</h2>

<p>Rodauth configuration generated by rodauth-rails provides several routes for
authentication and account management:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails rodauth:routes

<span class="c"># /login                   rodauth.login_path</span>
<span class="c"># /create-account          rodauth.create_account_path</span>
<span class="c"># /verify-account-resend   rodauth.verify_account_resend_path</span>
<span class="c"># /verify-account          rodauth.verify_account_path</span>
<span class="c"># /logout                  rodauth.logout_path</span>
<span class="c"># /remember                rodauth.remember_path</span>
<span class="c"># /reset-password-request  rodauth.reset_password_request_path</span>
<span class="c"># /reset-password          rodauth.reset_password_path</span>
<span class="c"># /change-password         rodauth.change_password_path</span>
<span class="c"># /change-login            rodauth.change_login_path</span>
<span class="c"># /verify-login-change     rodauth.verify_login_change_path</span>
<span class="c"># /close-account           rodauth.close_account_path</span>
</code></pre></div></div>

<p>Let’s use this information to add some main authentication links to our navbar:</p>

<div class="language-erb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- app/views/application/_navbar.html.erb --&gt;</span>
<span class="c">&lt;!-- ... ---&gt;</span>
<span class="cp">&lt;%</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">logged_in?</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"dropdown"</span><span class="nt">&gt;</span>
    <span class="nt">&lt;button</span> <span class="na">class=</span><span class="s">"btn btn-info dropdown-toggle"</span> <span class="na">data-bs-toggle=</span><span class="s">"dropdown"</span> <span class="na">type=</span><span class="s">"button"</span><span class="nt">&gt;</span>
      <span class="cp">&lt;%=</span> <span class="n">current_account</span><span class="p">.</span><span class="nf">email</span> <span class="cp">%&gt;</span>
    <span class="nt">&lt;/button&gt;</span>
    <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"dropdown-menu dropdown-menu-end"</span><span class="nt">&gt;</span>
      <span class="cp">&lt;%=</span> <span class="n">link_to</span> <span class="s2">"Change password"</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">change_password_path</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"dropdown-item"</span> <span class="cp">%&gt;</span>
      <span class="cp">&lt;%=</span> <span class="n">link_to</span> <span class="s2">"Change email"</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">change_login_path</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"dropdown-item"</span> <span class="cp">%&gt;</span>
      <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"dropdown-divider"</span><span class="nt">&gt;&lt;/div&gt;</span>
      <span class="cp">&lt;%=</span> <span class="n">link_to</span> <span class="s2">"Close account"</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">close_account_path</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"dropdown-item text-danger"</span> <span class="cp">%&gt;</span>
      <span class="cp">&lt;%=</span> <span class="n">link_to</span> <span class="s2">"Sign out"</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">logout_path</span><span class="p">,</span> <span class="ss">data: </span><span class="p">{</span> <span class="ss">turbo_method: :post</span> <span class="p">},</span> <span class="ss">class: </span><span class="s2">"dropdown-item"</span> <span class="cp">%&gt;</span>
    <span class="nt">&lt;/div&gt;</span>
  <span class="nt">&lt;/div&gt;</span>
<span class="cp">&lt;%</span> <span class="k">else</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;div&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">link_to</span> <span class="s2">"Sign in"</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">login_path</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"btn btn-outline-primary"</span> <span class="cp">%&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">link_to</span> <span class="s2">"Sign up"</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">create_account_path</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"btn btn-success"</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;/div&gt;</span>
<span class="cp">&lt;%</span> <span class="k">end</span> <span class="cp">%&gt;</span>
<span class="c">&lt;!-- ... ---&gt;</span>
</code></pre></div></div>

<p>Here we’re using the <code class="language-plaintext highlighter-rouge">#current_account</code> helper method that rodauth-rails
provides, which returns the currently signed in account.</p>

<p>Now our application will show login and registration links when the user is not
logged in:</p>

<p><img src="/images/rodauth-login-registration-links.png" alt="Rodauth login and registration links" /></p>

<p>While logged in users will see some basic account management links:</p>

<p><img src="/images/rodauth-account-management-links.png" alt="Rodauth account management links" /></p>

<h2 id="requiring-authentication">Requiring authentication</h2>

<p>Now that we have working authentication, we’ll likely want to require the user
to be authenticated for certain parts of our application. In our case, we want
to authenticate the posts controller.</p>

<p>We could add a <code class="language-plaintext highlighter-rouge">before_action</code> callback to the controller, but Rodauth allows
us to do this inside the Rodauth app’s route block, which is called before each
Rails route. This way we can keep our authentication logic contained in a
single place.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_app.rb</span>
<span class="k">class</span> <span class="nc">RodauthApp</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">App</span>
  <span class="c1"># ...</span>
  <span class="n">route</span> <span class="k">do</span> <span class="o">|</span><span class="n">r</span><span class="o">|</span>
    <span class="c1"># ...</span>
    <span class="k">if</span> <span class="n">r</span><span class="p">.</span><span class="nf">path</span><span class="p">.</span><span class="nf">start_with?</span><span class="p">(</span><span class="s2">"/posts"</span><span class="p">)</span>
      <span class="n">rodauth</span><span class="p">.</span><span class="nf">require_authentication</span>
    <span class="k">end</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Now visiting the <code class="language-plaintext highlighter-rouge">/posts</code> page will redirect the user to the <code class="language-plaintext highlighter-rouge">/login</code> page if
they’re not logged in.</p>

<p><img src="/images/rodauth-login-required.png" alt="Rodauth login required" /></p>

<p>We’ll also want to associate the posts to the <code class="language-plaintext highlighter-rouge">accounts</code> table:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate migration add_account_id_to_posts account:references
<span class="nv">$ </span>rails db:migrate
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/models/account.rb</span>
<span class="k">class</span> <span class="nc">Account</span> <span class="o">&lt;</span> <span class="no">ApplicationRecord</span>
  <span class="c1"># ...</span>
  <span class="n">has_many</span> <span class="ss">:posts</span>
<span class="k">end</span>
</code></pre></div></div>

<p>And scope them to the current account in the posts controller:</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/controllers/posts_controller.rb</span>
<span class="k">class</span> <span class="nc">PostsController</span> <span class="o">&lt;</span> <span class="no">ApplicationController</span>
  <span class="c1"># ...</span>
  <span class="k">def</span> <span class="nf">index</span>
    <span class="vi">@posts</span> <span class="o">=</span> <span class="n">current_account</span><span class="p">.</span><span class="nf">posts</span><span class="p">.</span><span class="nf">all</span>
  <span class="k">end</span>
  <span class="c1"># ...</span>
  <span class="k">def</span> <span class="nf">create</span>
    <span class="vi">@post</span> <span class="o">=</span> <span class="n">current_account</span><span class="p">.</span><span class="nf">posts</span><span class="p">.</span><span class="nf">build</span><span class="p">(</span><span class="n">post_params</span><span class="p">)</span>
    <span class="c1"># ...</span>
  <span class="k">end</span>
  <span class="c1"># ...</span>
  <span class="kp">private</span>
    <span class="k">def</span> <span class="nf">set_post</span>
      <span class="vi">@post</span> <span class="o">=</span> <span class="n">current_account</span><span class="p">.</span><span class="nf">posts</span><span class="p">.</span><span class="nf">find</span><span class="p">(</span><span class="n">params</span><span class="p">[</span><span class="ss">:id</span><span class="p">])</span>
    <span class="k">end</span>
    <span class="c1"># ...</span>
<span class="k">end</span>
</code></pre></div></div>

<h2 id="adding-new-fields">Adding new fields</h2>

<p>To have something other than an email address to display our users, let’s
require users to enter their name during registration. This will also give us
an opportunity to see how Rodauth can be configured.</p>

<p>Since we’ll need to edit the registration form, let’s first copy Rodauth’s HTML
templates into our Rails application:</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate rodauth:views

<span class="c"># create  app/views/rodauth/_login_form.html.erb</span>
<span class="c"># create  app/views/rodauth/_login_form_footer.html.erb</span>
<span class="c"># create  app/views/rodauth/_login_form_header.html.erb</span>
<span class="c"># create  app/views/rodauth/login.html.erb</span>
<span class="c"># create  app/views/rodauth/multi_phase_login.html.erb</span>
<span class="c"># create  app/views/rodauth/logout.html.erb</span>
<span class="c"># create  app/views/rodauth/create_account.html.erb</span>
<span class="c"># create  app/views/rodauth/verify_account_resend.html.erb</span>
<span class="c"># create  app/views/rodauth/verify_account.html.erb</span>
<span class="c"># create  app/views/rodauth/reset_password_request.html.erb</span>
<span class="c"># create  app/views/rodauth/reset_password.html.erb</span>
<span class="c"># create  app/views/rodauth/change_password.html.erb</span>
<span class="c"># create  app/views/rodauth/change_login.html.erb</span>
<span class="c"># create  app/views/rodauth/close_account.html.erb</span>
</code></pre></div></div>

<p>We can now open the <code class="language-plaintext highlighter-rouge">create_account.erb</code> template and add a new <code class="language-plaintext highlighter-rouge">name</code> field:</p>

<div class="language-erb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">&lt;!-- app/views/rodauth/create_account.erb --&gt;</span>
<span class="cp">&lt;%=</span> <span class="n">form_with</span> <span class="ss">url: </span><span class="n">rodauth</span><span class="p">.</span><span class="nf">create_account_path</span><span class="p">,</span> <span class="ss">method: :post</span><span class="p">,</span> <span class="ss">data: </span><span class="p">{</span> <span class="ss">turbo: </span><span class="kp">false</span> <span class="p">}</span> <span class="k">do</span> <span class="o">|</span><span class="n">form</span><span class="o">|</span> <span class="cp">%&gt;</span>
  <span class="c">&lt;!-- new "name" field --&gt;</span>
  <span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">"mb-3"</span><span class="nt">&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">form</span><span class="p">.</span><span class="nf">label</span> <span class="ss">:name</span><span class="p">,</span> <span class="s2">"Name"</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"form-label"</span> <span class="cp">%&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">form</span><span class="p">.</span><span class="nf">text_field</span> <span class="ss">:name</span><span class="p">,</span> <span class="ss">value: </span><span class="n">params</span><span class="p">[</span><span class="ss">:name</span><span class="p">],</span> <span class="ss">required: </span><span class="kp">true</span><span class="p">,</span> <span class="ss">class: </span><span class="s2">"form-control </span><span class="si">#{</span><span class="s2">"is-invalid"</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="s2">"name"</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span> <span class="ss">aria: </span><span class="p">({</span> <span class="ss">invalid: </span><span class="kp">true</span><span class="p">,</span> <span class="ss">describedby: </span><span class="s2">"login_error_message"</span> <span class="p">}</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="s2">"name"</span><span class="p">))</span> <span class="cp">%&gt;</span>
    <span class="cp">&lt;%=</span> <span class="n">content_tag</span><span class="p">(</span><span class="ss">:span</span><span class="p">,</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="s2">"name"</span><span class="p">),</span> <span class="ss">class: </span><span class="s2">"invalid-feedback"</span><span class="p">,</span> <span class="ss">id: </span><span class="s2">"login_error_message"</span><span class="p">)</span> <span class="k">if</span> <span class="n">rodauth</span><span class="p">.</span><span class="nf">field_error</span><span class="p">(</span><span class="s2">"name"</span><span class="p">)</span> <span class="cp">%&gt;</span>
  <span class="nt">&lt;/div&gt;</span>
  <span class="c">&lt;!-- ... --&gt;</span>
<span class="cp">&lt;%</span> <span class="k">end</span> <span class="cp">%&gt;</span>
</code></pre></div></div>

<p>Since the user’s name won’t be used for authentication, let’s store it in a new
<code class="language-plaintext highlighter-rouge">profiles</code> table, and associate the <code class="language-plaintext highlighter-rouge">profiles</code> table to the <code class="language-plaintext highlighter-rouge">accounts</code> table.</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>rails generate model Profile account:references name:string
<span class="nv">$ </span>rails db:migrate
</code></pre></div></div>
<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/models/account.rb</span>
<span class="k">class</span> <span class="nc">Account</span> <span class="o">&lt;</span> <span class="no">ApplicationRecord</span>
  <span class="c1"># ...</span>
  <span class="n">has_one</span> <span class="ss">:profile</span>
<span class="k">end</span>
</code></pre></div></div>

<p>We now need our Rodauth app to actually handle the new <code class="language-plaintext highlighter-rouge">name</code> parameter. We’ll
validate that it’s filled in and create the associated profile record after the
account is created.</p>

<div class="language-rb highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/misc/rodauth_main.rb</span>
<span class="k">class</span> <span class="nc">RodauthMain</span> <span class="o">&lt;</span> <span class="no">Rodauth</span><span class="o">::</span><span class="no">Rails</span><span class="o">::</span><span class="no">Auth</span>
  <span class="n">configure</span> <span class="k">do</span>
    <span class="c1"># ...</span>
    <span class="n">before_create_account</span> <span class="k">do</span>
      <span class="c1"># Validate presence of the name field</span>
      <span class="n">throw_error_status</span><span class="p">(</span><span class="mi">422</span><span class="p">,</span> <span class="s2">"name"</span><span class="p">,</span> <span class="s2">"must be present"</span><span class="p">)</span> <span class="k">unless</span> <span class="n">param_or_nil</span><span class="p">(</span><span class="s2">"name"</span><span class="p">)</span>
    <span class="k">end</span>
    <span class="n">after_create_account</span> <span class="k">do</span>
      <span class="c1"># Create the associated profile record with name</span>
      <span class="no">Profile</span><span class="p">.</span><span class="nf">create!</span><span class="p">(</span><span class="ss">account_id: </span><span class="n">account_id</span><span class="p">,</span> <span class="ss">name: </span><span class="n">param</span><span class="p">(</span><span class="s2">"name"</span><span class="p">))</span>
    <span class="k">end</span>
    <span class="n">after_close_account</span> <span class="k">do</span>
      <span class="c1"># Delete the associated profile record</span>
      <span class="no">Profile</span><span class="p">.</span><span class="nf">find_by!</span><span class="p">(</span><span class="ss">account_id: </span><span class="n">account_id</span><span class="p">).</span><span class="nf">destroy</span>
    <span class="k">end</span>
    <span class="c1"># ...</span>
  <span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>

<p>Now we can update our navbar to use the user’s name instead of their email
address:</p>

<div class="language-diff highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  &lt;button class="btn btn-info dropdown-toggle" data-bs-toggle="dropdown" type="button"&gt;
<span class="gd">-   &lt;%= current_account.email %&gt;
</span><span class="gi">+   &lt;%= current_account.profile.name %&gt;
</span>  &lt;/button&gt;
</code></pre></div></div>

<p><img src="/images/rodauth-account-name.png" alt="Displayed new account name" /></p>

<h2 id="closing-words">Closing words</h2>

<p>In this tutorial we’ve gradually built out a complete authentication and
account management flow using the Rodauth authentication framework. It
supports login &amp; logout, account creation with email verification and a grace
period, password change &amp; password reset, email change with email verification,
and close account functionality. We’ve seen how to add authentication links,
require authentication for certain routes, and add new fields to the registration
form.</p>

<p>I’m personally very excited about Rodauth, as it has an impressive featureset
and a refreshingly clean design, and also it’s not tied to Rails. I’ve been
working hard on <a href="https://github.com/janko/rodauth-rails">rodauth-rails</a> to make it as easy as possible to get started
with in Rails, so hopefully it will help Rodauth gain more traction in the
Rails community.</p>]]></content><author><name>Janko Marohnić</name><email>janko@hey.com</email></author><category term="rodauth" /><summary type="html"><![CDATA[In this tutorial, we’ll show how to add fully functional authentication and account management functionality into a Rails app, using the Rodauth authentication framework. Rodauth has many advantages over the mainstream alternatives such as Devise, Sorcery, Clearance, and Authlogic, see my previous article for an introduction.]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://janko.io/images/social.jpg" /><media:content medium="image" url="https://janko.io/images/social.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry></feed>