Jekyll2022-10-16T17:26:31+02:00https://mergujmyniktniewola.it/feed.xmlMergujmy, nikt nie woła!Hi, this is my simple blog about my adventures in software-development-land. Lets merge, nobody calls!
Don’t DRY your specs2020-11-18T00:00:00+01:002020-11-18T00:00:00+01:00https://mergujmyniktniewola.it/2020/11/18/dont-dry-your-specs<p>I read pull requests every day. I read much more pull requests that I create. I am often amazed how
specs are not treated as first class citizens in project. Ofcourse every engineer will said it is
not true because he sees the value in specs. But, do they?</p>
<p>Every programmer tries to craft his codebase to keep it clean, robust and well structured. But in
some degree of limitation. All the application code is always clean, developers knows tons of great
patterns, conventions etc. to keep application code clean. Sometimes even it is really clean! But
when I look into specs … man, most of them look like created using the potato stamp. Mixed,
complicated, long, hard to understand after some time … But after all they are green, no failure,
green dots in out. Why bother to something more?</p>
<p>Consider this simple, almost pseudo-code rspec example:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">describe</span> <span class="no">SomeCleanClass</span> <span class="k">do</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:dependency_a</span><span class="p">)</span> <span class="p">{</span> <span class="no">DependencyA</span><span class="p">.</span><span class="nf">new</span> <span class="p">}</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:dependenby_b</span><span class="p">)</span> <span class="p">{</span> <span class="no">DependencyB</span><span class="p">.</span><span class="nf">new</span> <span class="p">}</span>
<span class="n">subject</span> <span class="p">{</span> <span class="n">described_class</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">dependency_a</span><span class="p">,</span> <span class="n">dependency_b</span><span class="p">)</span> <span class="p">}</span>
<span class="n">describe</span> <span class="s1">'#method_a'</span> <span class="k">do</span>
<span class="c1"># testing the method_a goes here</span>
<span class="n">context</span> <span class="s1">'when something something'</span> <span class="k">do</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:dependency_a</span><span class="p">)</span> <span class="p">{</span> <span class="no">SomeSpecificCaseDependencyA</span><span class="p">.</span><span class="nf">new</span> <span class="p">}</span>
<span class="k">end</span>
<span class="n">context</span> <span class="s1">'when something other'</span> <span class="k">do</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">describe</span> <span class="s1">'#method_b'</span> <span class="k">do</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:dependency_a</span><span class="p">)</span> <span class="p">{</span> <span class="no">SomeOtherDependencyA</span><span class="p">.</span><span class="nf">new</span> <span class="p">}</span>
<span class="c1"># testing the method_b goes here</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>There is one particular thing I would like to bring to your attention. This test is optimized for
code duplication, not code readability. After all why to bother? It produces a green, little dot in
output. Well … code is read much more often than written. Same applies to specs, maybe even in
more degree. Often I see that developers struggle to generalize tests source code because repetition
is a source of all evil of 2020 year! Don’t repeat yourself, they said. Sometimes I even think that
there are developers with DRY tatoo!</p>
<p>In above example it is not so visible. rspec is a great tool for testing, it also has a ability to
generate lines of code like creazy. When you test object with two or three methods it is easy to
product more than one screen of code. You are working with one method spec and constantly you are
jumping to the top of file to check definition, then check the overidden values. Mindblow!</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">describe</span> <span class="no">SomeCleanClass</span> <span class="k">do</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:dependency_a</span><span class="p">)</span> <span class="p">{</span> <span class="no">DependencyA</span><span class="p">.</span><span class="nf">new</span> <span class="p">}</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:dependenby_b</span><span class="p">)</span> <span class="p">{</span> <span class="no">DependencyB</span><span class="p">.</span><span class="nf">new</span> <span class="p">}</span>
<span class="n">subject</span> <span class="p">{</span> <span class="n">described_class</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">dependency_a</span><span class="p">,</span> <span class="n">dependency_b</span><span class="p">)</span> <span class="p">}</span>
<span class="n">describe</span> <span class="s1">'#method_a'</span> <span class="k">do</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:dependency_a</span><span class="p">)</span> <span class="p">{</span> <span class="no">DependencyA</span><span class="p">.</span><span class="nf">new</span> <span class="p">}</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:dependenby_b</span><span class="p">)</span> <span class="p">{</span> <span class="no">DependencyB</span><span class="p">.</span><span class="nf">new</span> <span class="p">}</span>
<span class="n">subject</span> <span class="p">{</span> <span class="n">described_class</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">dependency_a</span><span class="p">,</span> <span class="n">dependency_b</span><span class="p">)</span> <span class="p">}</span>
<span class="n">context</span> <span class="s1">'when something something'</span> <span class="k">do</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:dependency_a</span><span class="p">)</span> <span class="p">{</span> <span class="no">SomeSpecificCaseDependencyA</span><span class="p">.</span><span class="nf">new</span> <span class="p">}</span>
<span class="k">end</span>
<span class="n">context</span> <span class="s1">'when something other'</span> <span class="k">do</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">describe</span> <span class="s1">'#method_b'</span> <span class="k">do</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:dependency_a</span><span class="p">)</span> <span class="p">{</span> <span class="no">DependencyA</span><span class="p">.</span><span class="nf">new</span> <span class="p">}</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:dependenby_b</span><span class="p">)</span> <span class="p">{</span> <span class="no">DependencyB</span><span class="p">.</span><span class="nf">new</span> <span class="p">}</span>
<span class="n">subject</span> <span class="p">{</span> <span class="n">described_class</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">dependency_a</span><span class="p">,</span> <span class="n">dependency_b</span><span class="p">)</span> <span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>I read pull requests every day. I read much more pull requests that I create. I am often amazed how specs are not treated as first class citizens in project. Ofcourse every engineer will said it is not true because he sees the value in specs. But, do they?Building evolutionary architectures. Review.2020-08-10T00:00:00+02:002020-08-10T00:00:00+02:00https://mergujmyniktniewola.it/2020/08/10/building-evolutionary-architectures-review<p>Recently I read the <a href="https://www.amazon.com/Building-Evolutionary-Architectures-Support-Constant-ebook/dp/B075RR1XVG/ref=sr_1_1?crid=G7DRR3M01K9I&dchild=1&keywords=building+evolutionary+architectures&_encoding=UTF8&tag=seban-20&linkCode=ur2&linkId=45992a0e8d6006c9952c9cac75ebed82&camp=1789&creative=9325">Building Evolutionary Architectures: Support Constant
Change</a>
book by Rebecca Parsons and Patrick Kua. I decided to read this title after a recommendation from
<a href="https://lethain.com/about/">Will Larson blogpost</a>.</p>
<p>I had a high expectation for this book. I believe that this book will be kind of an eye-opener for
me and I will get some deep knowledge insights. Am I disappointed by it? Yes, a little bit. But this
does not mean that this book is a scam.</p>
<p>Authors spend many pages explaining the idea of fitness functions. I am taking it as a whole. Realy.
I like it and already think which fitness function I can apply to projects I work in. What I miss is
the catalogue of fitness functions grouped by category. Some place where I can get inspiration for
defining my own fitness functions.</p>
<p>I am sure that less experienced software engineers will find useful knowledge and principles. There
is a great overview of a couple of most used architecture types! There are plenty of moments when
authors remind what is crucial for modern business - time to market and the possibility to change
directions when needed. This book can be a good entry step into the topic of software architectures.
If you are looking for some materials I recommend you to visit
<a href="https://evolutionaryarchitecture.com">https://evolutionaryarchitecture.com</a> page. There are links
to various materials about this topic, videos, podcasts, etc. There is even the section with
<a href="http://evolutionaryarchitecture.com/ffkatas/index.html">katas</a> which my by interesting team
activity!</p>Recently I read the Building Evolutionary Architectures: Support Constant Change book by Rebecca Parsons and Patrick Kua. I decided to read this title after a recommendation from Will Larson blogpost.Elegant Puzzle book review2020-04-10T00:00:00+02:002020-04-10T00:00:00+02:00https://mergujmyniktniewola.it/2020/04/10/elegant-puzzle-book-review<p>I don’t remember how I get to Will Larson <a href="https://lethain.com/">Irrational Exuberance</a> blog. But I
remember that for sure that I need to read more than one blog post. While reading second or third, I
started to wonder who is the author. On About page, I read Will Larson worked in
<a href="https://uber.com">Uber</a>, <a href="https://digg.com">Digg</a> and <a href="https://stripe.com">Stripe</a>. Top-notch
companies. Moreover, I get to know that he wrote a book <a href="https://amzn.to/2RpkdRy">An Elegant Puzzle: Systems of Engineering
Management</a>. I decided that I should read it.</p>
<p align="center">
<a target="_blank" href="https://www.amazon.com/gp/product/1732265186/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1732265186&linkCode=as2&tag=seban-20&linkId=c16526f87a7185f7bbb79f9be57b3bea"><img border="0" src="//ws-na.amazon-adsystem.com/widgets/q?_encoding=UTF8&MarketPlace=US&ASIN=1732265186&ServiceVersion=20070822&ID=AsinImage&WS=1&Format=_SL250_&tag=seban-20" /></a><img src="//ir-na.amazon-adsystem.com/e/ir?t=seban-20&l=am2&o=1&a=1732265186" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />
</p>
<p>A couple of weeks later I bought a copy and sent it to my <a href="https://amzn.to/2wqxNwu">Kindle</a> reader.
After the first couple of pages, I noticed approaches that I can apply directly to my team. On later
parts of the <a href="https://amzn.to/2RpkdRy">Elegant Puzzle</a>, many things don’t apply to me. Mainly
because it was about higher management, managing managers or even executives. I like the form of the
book. Many topics, problem solutions are described as lists in points you should follow. It is
helpful, but simple! <a target="_blank" href="https://www.amazon.com/Will-Larson/e/B07RCW6CWQ/ref=dp_byline_cont_book_1?&_encoding=UTF8&tag=seban-20&linkCode=ur2&linkId=7d2b966f21d04985ff2ce763dae559f5&camp=1789&creative=9325">The author</a><img src="//ir-na.amazon-adsystem.com/e/ir?t=seban-20&l=ur2&o=1" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /> relies mostly on his
personal experience at successful tech companies.</p>
<p>A great part of the book is <em>Appendix</em>. You will find a list of books and technical papers
recommended by the author. I must admit that a couple of books recommended by Larson were added to
my wish list after reading a book.</p>
<p>I recommend <a href="https://amzn.to/2RpkdRy">this book</a> to all folks working on engineering management
positions. It is a very useful position for all working in rapidly growing companies. You will find
a detailed prescription for forming your teams, reorganize the structure and hire engineers.</p>I don’t remember how I get to Will Larson Irrational Exuberance blog. But I remember that for sure that I need to read more than one blog post. While reading second or third, I started to wonder who is the author. On About page, I read Will Larson worked in Uber, Digg and Stripe. Top-notch companies. Moreover, I get to know that he wrote a book An Elegant Puzzle: Systems of Engineering Management. I decided that I should read it.SQL window functions OVER PARTITION2020-04-03T00:00:00+02:002020-04-03T00:00:00+02:00https://mergujmyniktniewola.it/2020/04/03/sql-window-functions-over-partition<p>As I wrote earlier in the post about <a href="/2017/12/17/sql-filter-clause.html">SQL filter clause</a> I like SQL. It is so compact, with a small amount of code, great results can be achieved. Once for some time, I have a revelation and I can write a query using so-called modern techniques. Recently I used <code class="highlighter-rouge">OVER PARTITION BY</code>.</p>
<p>This SQL feature is related to <a href="https://www.postgresql.org/docs/current/tutorial-window.html">Window Functions</a>. It allows you to perform a calculation on other rows that are somehow connected to the current row. Seems cryptic, right?</p>
<p>So imagine that you have <code class="highlighter-rouge">purchases</code> and those purchases have a <code class="highlighter-rouge">price</code>, <code class="highlighter-rouge">name</code> of the sold product and date of transaction.</p>
<table>
<thead>
<tr>
<th>name</th>
<th style="text-align: right">price</th>
<th>created_at</th>
</tr>
</thead>
<tbody>
<tr>
<td>ball</td>
<td style="text-align: right">9.99</td>
<td>2020-03-30 14:00</td>
</tr>
<tr>
<td>hat</td>
<td style="text-align: right">19.99</td>
<td>2020-03-30 15:10</td>
</tr>
<tr>
<td>ball</td>
<td style="text-align: right">9.99</td>
<td>2020-03-31 04:14</td>
</tr>
<tr>
<td>ball</td>
<td style="text-align: right">9.99</td>
<td>2020-03-31 10:45</td>
</tr>
<tr>
<td>hat</td>
<td style="text-align: right">9.99</td>
<td>2020-03-31 16:08</td>
</tr>
<tr>
<td>cup</td>
<td style="text-align: right">4.99</td>
<td>2020-03-31 19:36</td>
</tr>
</tbody>
</table>
<p>When you need to know how much you earn each day, you will execute the SQL query with simple aggregation.</p>
<pre><code class="language-SQL">SELECT created_at::date AS day, SUM(price)
FROM purchases
GROUP BY 1;
</code></pre>
<table>
<thead>
<tr>
<th>day</th>
<th style="text-align: right">sum</th>
</tr>
</thead>
<tbody>
<tr>
<td>2020-03-30</td>
<td style="text-align: right">29.98</td>
</tr>
<tr>
<td>2020-03-31</td>
<td style="text-align: right">34.96</td>
</tr>
</tbody>
</table>
<p>IF you need to know how much balls, hats and mugs you sold and how much you earn each day?</p>
<pre><code class="language-SQL">SELECT created_at::date AS day, name, COUNT(*), SUM(price)
FROM purchases
GROUP BY 1, 2;
</code></pre>
<table>
<thead>
<tr>
<th>day</th>
<th>name</th>
<th style="text-align: right">count</th>
<th style="text-align: right">sum</th>
</tr>
</thead>
<tbody>
<tr>
<td>2020-03-30</td>
<td>ball</td>
<td style="text-align: right">1</td>
<td style="text-align: right">9.99</td>
</tr>
<tr>
<td>2020-03-30</td>
<td>hat</td>
<td style="text-align: right">1</td>
<td style="text-align: right">19.99</td>
</tr>
<tr>
<td>2020-03-31</td>
<td>ball</td>
<td style="text-align: right">2</td>
<td style="text-align: right">19.98</td>
</tr>
<tr>
<td>2020-03-31</td>
<td>hat</td>
<td style="text-align: right">1</td>
<td style="text-align: right">19.99</td>
</tr>
<tr>
<td>2020-03-31</td>
<td>cup</td>
<td style="text-align: right">1</td>
<td style="text-align: right">4.99</td>
</tr>
</tbody>
</table>
<p>Cool, that was easy. What a great data analyst you are! ;-)</p>
<p>Now lets add one more column to above result - sum of daily all earnings, no matter which product was sold.</p>
<pre><code class="language-SQL">SELECT
created_at::date AS day,
name,
COUNT(*),
SUM(price),
SUM(SUM(price)) OVER (PARTITION BY created_at::date) AS daily_sum
FROM purchases
GROUP BY 1, 2;
</code></pre>
<p>Result will look like:</p>
<table>
<thead>
<tr>
<th>day</th>
<th>name</th>
<th style="text-align: right">count</th>
<th style="text-align: right">sum</th>
<th style="text-align: right">daily_sum</th>
</tr>
</thead>
<tbody>
<tr>
<td>2020-03-30</td>
<td>ball</td>
<td style="text-align: right">1</td>
<td style="text-align: right">9.99</td>
<td style="text-align: right">29.98</td>
</tr>
<tr>
<td>2020-03-30</td>
<td>hat</td>
<td style="text-align: right">1</td>
<td style="text-align: right">19.99</td>
<td style="text-align: right">29.98</td>
</tr>
<tr>
<td>2020-03-31</td>
<td>ball</td>
<td style="text-align: right">2</td>
<td style="text-align: right">19.98</td>
<td style="text-align: right">44.96</td>
</tr>
<tr>
<td>2020-03-31</td>
<td>hat</td>
<td style="text-align: right">1</td>
<td style="text-align: right">19.99</td>
<td style="text-align: right">44.96</td>
</tr>
<tr>
<td>2020-03-31</td>
<td>cup</td>
<td style="text-align: right">1</td>
<td style="text-align: right">4.99</td>
<td style="text-align: right">44.96</td>
</tr>
</tbody>
</table>
<p>It it is not ideal:</p>
<ul>
<li><code class="highlighter-rouge">daily_sum</code> is duplicated acros rows with the same <code class="highlighter-rouge">day</code></li>
<li>this <code class="highlighter-rouge">SUM(SUM(price))</code> looks ugly</li>
</ul>
<p>For sure there is some more elegant solution for such use case. But this one suits my needs perfectly.</p>
<ul>
<li>I got the sum & count grouped by day and product name</li>
<li>I got the daily sum grouped by day</li>
<li>all is in one result set</li>
</ul>
<p><code class="highlighter-rouge">OVER</code> can be also used without <code class="highlighter-rouge">GROUP BY</code>. Over internet you can find plenty of examples with employees salary. If you need choose then go with <code class="highlighter-rouge">GROUP BY</code> as it is probably better optimized in your database.</p>
<p>I found it really handy when you need to perform some kind of aggregation but you don’t want to loose any rows from your dataset.</p>As I wrote earlier in the post about SQL filter clause I like SQL. It is so compact, with a small amount of code, great results can be achieved. Once for some time, I have a revelation and I can write a query using so-called modern techniques. Recently I used OVER PARTITION BY.My first experience with AWS Step Functions2020-02-10T00:00:00+01:002020-02-10T00:00:00+01:00https://mergujmyniktniewola.it/2020/02/10/my-first-experience-with-aws-step-functions<p>Step functions are the way to coordinate components of a distributed application. Every component performs a well defined, separated task where output becomes an input of other tasks. Step functions consist of State Machines.</p>
<h3 id="amazon-states-language">Amazon States Language</h3>
<p>Amazon State Language is JSON-based language for defining state machines used by Step Functions. It is a collection of states leading execution from start state (can be only one) to very last state (many states can be the last one). Example of ASL</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"Comment"</span><span class="p">:</span><span class="w"> </span><span class="s2">"First state machine"</span><span class="p">,</span><span class="w">
</span><span class="nl">"StartAt"</span><span class="p">:</span><span class="w"> </span><span class="s2">"HelloSt"</span><span class="p">,</span><span class="w">
</span><span class="nl">"States"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"HelloSt"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"Type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Pass"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Result"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hello"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Next"</span><span class="p">:</span><span class="w"> </span><span class="s2">"World"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"WorldSt"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"Type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Pass"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Result"</span><span class="p">:</span><span class="w"> </span><span class="s2">"World"</span><span class="p">,</span><span class="w">
</span><span class="nl">"End"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>Above state machine will start on <code class="highlighter-rouge">HelloSt</code> state and pass result <code class="highlighter-rouge">"Hello"</code> to the <code class="highlighter-rouge">WorldSt</code> state input. Below is the graphic representation of this ASL:</p>
<p><img src="/images/aws-step-fuctions/hello-world-state.png" alt="Hello World state" /></p>
<p>Conditional statements are also supported:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"Comment"</span><span class="p">:</span><span class="w"> </span><span class="s2">"First state machine"</span><span class="p">,</span><span class="w">
</span><span class="nl">"StartAt"</span><span class="p">:</span><span class="w"> </span><span class="s2">"CheckName"</span><span class="p">,</span><span class="w">
</span><span class="nl">"States"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"CheckName"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"Type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Choice"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Choices"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"Not"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"Variable"</span><span class="p">:</span><span class="w"> </span><span class="s2">"$.name"</span><span class="p">,</span><span class="w">
</span><span class="nl">"StringEquals"</span><span class="p">:</span><span class="w"> </span><span class="s2">""</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"Next"</span><span class="p">:</span><span class="w"> </span><span class="s2">"HelloName"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"Variable"</span><span class="p">:</span><span class="w"> </span><span class="s2">"$.name"</span><span class="p">,</span><span class="w">
</span><span class="nl">"StringEquals"</span><span class="p">:</span><span class="w"> </span><span class="s2">""</span><span class="p">,</span><span class="w">
</span><span class="nl">"Next"</span><span class="p">:</span><span class="w"> </span><span class="s2">"HelloWorld"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"HelloName"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"Type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Pass"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Result"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hello"</span><span class="p">,</span><span class="w">
</span><span class="nl">"End"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"HelloWorld"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"Type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Pass"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Result"</span><span class="p">:</span><span class="w"> </span><span class="s2">"World"</span><span class="p">,</span><span class="w">
</span><span class="nl">"End"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>First state <code class="highlighter-rouge">CheckName</code> will decide, based on the <code class="highlighter-rouge">name</code> input parameter, to which next state flow should be directed. Above code will produce state machine:</p>
<p><img src="/images/aws-step-fuctions/conditional-hello-world-state.png" alt="Conditional Hello World state" /></p>
<p>For more information visit the official documentation of <a href="https://docs.aws.amazon.com/step-functions/latest/dg/concepts-amazon-states-language.html">Amazon States Language</a>.</p>
<h3 id="the-use-case-i-was-interested-most">The use case I was interested most</h3>
<p>I was most interested in a use case like</p>
<blockquote>
<p>Send an email to the customer after defined time passed since some fact occurs</p>
</blockquote>
<p>I see a nice use case for <code class="highlighter-rouge">Wait</code> state type here. <a href="https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-wait-state.html">Wait state</a> delays state machine from continuing for a specified time - it can be a defined number of seconds, fixed timestamp, it can also be passed as a call parameter. The state machine will be very simple</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"Comment"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Send an incentive email"</span><span class="p">,</span><span class="w">
</span><span class="nl">"StartAt"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Wait"</span><span class="p">,</span><span class="w">
</span><span class="nl">"States"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"Wait"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"Type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Wait"</span><span class="p">,</span><span class="w">
</span><span class="nl">"TimestampPath"</span><span class="p">:</span><span class="w"> </span><span class="s2">"$.scheduledAt"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Next"</span><span class="p">:</span><span class="w"> </span><span class="s2">"SendEmail"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"SendEmail"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"Type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Task"</span><span class="p">,</span><span class="w">
</span><span class="nl">"Resource"</span><span class="p">:</span><span class="w"> </span><span class="s2">"arn:aws:lambda:eu-central-1:074085690123:function:send-incentive-email"</span><span class="p">,</span><span class="w">
</span><span class="nl">"End"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p><img src="/images/aws-step-fuctions/send-incentive-email-state.png" alt="Send incentive email state machine" /></p>
<p>I choose a way to provide the exact time when <code class="highlighter-rouge">WaitForSchedule</code> state should pass execution further to <code class="highlighter-rouge">SendEmail</code> state by using <code class="highlighter-rouge">TimestampPath</code> attribute in state definition. Now I can invoke a state machine with an example event</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"scheduledAt"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2019-12-31T08:13:00Z"</span><span class="p">,</span><span class="w"> </span><span class="err">#</span><span class="w"> </span><span class="err">This</span><span class="w"> </span><span class="err">is</span><span class="w"> </span><span class="err">when</span><span class="w"> </span><span class="err">state</span><span class="w"> </span><span class="err">will</span><span class="w"> </span><span class="err">transit</span><span class="w"> </span><span class="err">to</span><span class="w"> </span><span class="err">`SendEmail`</span><span class="w">
</span><span class="nl">"locale"</span><span class="p">:</span><span class="w"> </span><span class="s2">"en"</span><span class="p">,</span><span class="w">
</span><span class="nl">"airline"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Lufthansa"</span><span class="p">,</span><span class="w">
</span><span class="nl">"departure_airport"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Frankfurt"</span><span class="p">,</span><span class="w">
</span><span class="nl">"arrival_airport"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Gdańsk"</span><span class="p">,</span><span class="w">
</span><span class="nl">"compensation"</span><span class="p">:</span><span class="w"> </span><span class="s2">"600"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>Example execution trigger can be implemented, with an official SDK, like below</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'aws-sdk-states'</span>
<span class="nb">require</span> <span class="s1">'json'</span>
<span class="n">client</span> <span class="o">=</span> <span class="no">Aws</span><span class="o">::</span><span class="no">States</span><span class="o">::</span><span class="no">Client</span><span class="p">.</span><span class="nf">new</span>
<span class="c1"># assuming there is only one state machine defined</span>
<span class="n">state_machine_arn</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">list_state_machines</span><span class="p">.</span><span class="nf">state_machines</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nf">state_machine_arn</span>
<span class="n">client</span><span class="p">.</span><span class="nf">start_execution</span><span class="p">({</span>
<span class="ss">state_machine_arn: </span><span class="n">state_machine_arn</span><span class="p">,</span>
<span class="ss">input: </span><span class="p">{</span>
<span class="ss">scheduledAt: </span><span class="p">(</span><span class="no">Time</span><span class="p">.</span><span class="nf">now</span><span class="p">.</span><span class="nf">utc</span> <span class="o">+</span> <span class="mi">60</span><span class="p">).</span><span class="nf">strftime</span><span class="p">(</span><span class="s1">'%Y-%m-%dT%H:%M:%SZ'</span><span class="p">),</span>
<span class="ss">locale: </span><span class="s2">"en"</span><span class="p">,</span>
<span class="ss">airline: </span><span class="s2">"Lufthansa"</span><span class="p">,</span>
<span class="ss">departure_airport: </span><span class="s2">"Frankfurt"</span><span class="p">,</span>
<span class="ss">arrival_airport: </span><span class="s2">"Gdańsk"</span><span class="p">,</span>
<span class="ss">compensation: </span><span class="mi">600</span>
<span class="p">}.</span><span class="nf">to_json</span>
<span class="p">})</span>
</code></pre></div></div>
<p>Execution can be named. Such name must be unique in the scope of state machine - sounds like a good way to achieve exactly one execution.
<code class="highlighter-rouge">SendEmail</code> state can be extended by <code class="highlighter-rouge">Retry</code> (max attempts and exponential backoff supported!) and <code class="highlighter-rouge">Catch</code> attributes. Both limited to particular <code class="highlighter-rouge">ErrorType</code>.</p>
<h3 id="pricing">Pricing</h3>
<p>Free Tier includes 4000 state transitions. To make it simple, it is a count of edges between states. Sending delayed email required three state transitions to perform. Every next 1000 state transition costs $0.025.</p>
<h3 id="summary">Summary</h3>
<p>Step functions look quite interesting, they were introduced in December 2016 and I haven’t played with them since today. Step function may look very limited and simple at first, but I can imagine some big state machines orchestrating complex execution logic.</p>
<h3 id="state-machine-can-be-executed">State machine can be executed</h3>
<ul>
<li>via API action (like in the above example)</li>
<li>CloudWatch events (haven’t tried)</li>
<li>Amazon API Gateway (abstraction over abstraction?)</li>
<li>from other State Machine (we need to go deeper …)</li>
</ul>
<h3 id="what-state-machine-can-play-with">What state machine can play with</h3>
<ul>
<li>Lambda functions (like in the above example)</li>
<li>DymamoDB (read & write)</li>
<li>SNS (publishing message)</li>
<li>SQS (putting a message into the queue)</li>
<li>some other AWS services I haven’t heard of</li>
</ul>
<h3 id="where-i-find-it-useful">Where I find it useful</h3>
<ul>
<li>data processing and ETLs</li>
<li>delaying Lambda execution (like in the above example)</li>
<li>kind of continuous integration with Activities?</li>
</ul>
<h3 id="what-i-havent-touched">What I haven’t touched</h3>
<p>State function offers also tasks type <code class="highlighter-rouge">Parallel</code> and <code class="highlighter-rouge">Map</code>. First can execute many tasks at once, for example sending email and sending SMS at the same time (duration savings). Second can execute the same step for every item in input array concurrently (duration and state transition savings).</p>
<p>All of “real executors” - <code class="highlighter-rouge">Task</code>, <code class="highlighter-rouge">Parallel</code> and <code class="highlighter-rouge">Map</code> offers the <code class="highlighter-rouge">Catch</code> and <code class="highlighter-rouge">Retry</code> options. In my opinion, <code class="highlighter-rouge">Catch</code> should be used more like workflow fallback in case of error instead of handling errors inside Lambda Function. <code class="highlighter-rouge">Retry</code> is … well, retry.</p>
<p>Every step in workflow and workflow itself can have defined limit after which move error (no output from state/workflow) or to a fallback defined state.</p>
<p>Any logging capabilities of workflows execution.</p>
<p>Activities - special kind of workflows that allow you to execute any task on Amazon EC2, Amazon Elastic Container Service, mobile device, …</p>
<h3 id="pros">Pros</h3>
<ul>
<li>relatively easy to start</li>
<li><a href="https://www.terraform.io/">terraform</a> support</li>
<li>simple and express workflows</li>
<li>maximum execution time for workflow is 1 year</li>
<li><a href="https://aws.amazon.com/about-aws/whats-new/2019/12/introducing-aws-step-functions-express-workflows/">Express workflows</a> looks fast, according to documentation event rates greater than 100,000 events per second</li>
<li>tooling inside AWS Console is nice, allow to execute and check when which state was executed with inputs and outputs</li>
<li>Lambda functions can be independent, without dependency on each other, without referencing other resources, pure input and output</li>
</ul>
<h3 id="cons">Cons</h3>
<ul>
<li>pricing for complex and long-running workflows</li>
<li>can’t be executed by SNS event</li>
<li>workflow source code inside Terraform</li>
<li>while there is some linter for Amazon State Language, I find missing test tool as a drawback</li>
<li>lack of scheduling vs. scheduled Lambda functions</li>
</ul>Step functions are the way to coordinate components of a distributed application. Every component performs a well defined, separated task where output becomes an input of other tasks. Step functions consist of State Machines.Help yourself to write good commit message2019-12-03T00:00:00+01:002019-12-03T00:00:00+01:00https://mergujmyniktniewola.it/2019/12/03/help-yourself-to-write-good-commit-message<p>The problem of poor or even shitty commit messages backs to me regularly like a boomerang. I can evangelize, I can point the good examples, the bad examples but it still comes back. Why it is happening?</p>
<p>It shouldn’t be if you have some self discipline in you. A good commit message is the beginning of the craftsmanship way for any software professional. But most of the software developers are lazy and they follow DRY principle because it sounds so easy, when firing up <code class="highlighter-rouge">git commit -m "Fix"</code> or <code class="highlighter-rouge">git commit -m "Post review fixes"</code>
Or … Probably every repository has at least dozen of such examples.</p>
<p>The truth is that <code class="highlighter-rouge">git</code> gives us tools to write good commit messages. Even official documentation for <a href="https://git-scm.com/docs/git-commit#_discussion">git-commit</a> gives some pieces of advice. But today I would like to say about a specific switch of <code class="highlighter-rouge">git-commit</code>. <code class="highlighter-rouge">--verbose</code> option is something you should look for. Instead of using <code class="highlighter-rouge">-m</code> option to provide meaningless message use <code class="highlighter-rouge">--verbose</code>. This will allow you to provide a great message in your favourite text editor but at the bottom of message template, you will see the diff between your changes and <code class="highlighter-rouge">HEAD</code>. See the image below (Emacs in my case)</p>
<p><img src="/images/git-verbose-emacs.png" alt="git commit --verbose in Emacs" /></p>
<p>A couple of months ago I added this to my git-config</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git config commit.verbose true
</code></pre></div></div>
<p>This will make your life easier and you will be seen as true professional by other software developers.</p>The problem of poor or even shitty commit messages backs to me regularly like a boomerang. I can evangelize, I can point the good examples, the bad examples but it still comes back. Why it is happening?The Goal book review2019-02-11T00:00:00+01:002019-02-11T00:00:00+01:00https://mergujmyniktniewola.it/2019/02/11/the-goal-book-review<p>It is one more book review after <a href="/2017/12/09/devops-handbook.html">The DevOps
handbook</a>, <a href="/2018/08/06/the-phoenix-project-book-review.html">The Phoenix
Project</a>, <a href="/2019/01/01/accelerate-book-review.html">Them
Accelerate</a> on my blog. <a href="https://amzn.to/2RfSCCe">“The
Goal” by Goldratt</a> is like a natural next step after reading the
mentioned books.</p>
<p>Professionally I have nothing to do with plants and any hardware
industry, I am connected with software development. But over fifteen
years ago I worked in a quite big plant as a warehouse worker and I
saw industry from the inside. This experience probably allowed me to
better understand some problems Alex Rogo and his gang must to deal
with. In my opinion, those problems and consequences are described in
easy to understand form. The title “The Goal” is explained very
shortly after starting the book and I must admit it was not clear to
me what it is before reaching this page of the book. Spoiler alert!
The goal of every company is to make money. Nothing less, nothing
more. Simple, right?</p>
<p>On later parts of book <a href="https://en.wikipedia.org/wiki/Theory_of_constraints">Theory of
Constraints</a> is
formed together with five rules on how to deal with bottlenecks. These
rules are so common that should fit into every industry, all you need
is only to find a connection between them and your environment. The
metaphor of the working process and scout hike was brilliant, showing
all its nuances and complexity of possible obstacles.</p>
<p>I liked also the “personal thread” of Alex Rogo family. His personal
problems should induce reflection in every reader about work-life
balance. I found confirmation in <a href="https://amzn.to/2RfSCCe">“The Goal”</a> that some of the
techniques I used in the last couple of months was right and it should
be continued. That is very rewarding.</p>
<p>I recommend this book to all interested in process optimization and
managers. If you are working, like me, in software development I think
you should choose The Phoenix Project which is very much like The Goal
but more focused on IT.</p>It is one more book review after The DevOps handbook, The Phoenix Project, Them Accelerate on my blog. “The Goal” by Goldratt is like a natural next step after reading the mentioned books.Accelerate book review2019-01-01T00:00:00+01:002019-01-01T00:00:00+01:00https://mergujmyniktniewola.it/2019/01/01/accelerate-book-review<p>First of all lets write the full title <a href="https://amzn.to/3bT4F01">Accelerate: The Science of
Lean Software and DevOps: Building and Scaling High Performing
Technology Organizations</a>. What nice book it is! I took this book to
train while commuting between Gdańsk and Cracow, together with my pal
<a href="https://k85.pl">Tomek, The Agile Guy</a>, some time ago and this book was a superb
travel companion.</p>
<p align="center">
<a target="_blank" href="https://www.amazon.com/gp/product/1942788339/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=1942788339&linkCode=as2&tag=seban-20&linkId=a7e78d6ebf5d7857e90dde36af23759e"><img border="0" src="https://ws-na.amazon-adsystem.com/widgets/q?_encoding=UTF8&MarketPlace=US&ASIN=1942788339&ServiceVersion=20070822&ID=AsinImage&WS=1&Format=_SL250_&tag=seban-20" /></a><img src="https://ir-na.amazon-adsystem.com/e/ir?t=seban-20&l=am2&o=1&a=1942788339" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />
</p>
<p>This book is like a scientific paper on software development and
delivery. I learnt so many wise words and topics while reading it, not
only about software development. Inside the book, you will find plenty
of statements like</p>
<blockquote>
<p>‘teams practising A can expect a change in metric B by X%’.</p>
</blockquote>
<p>Moreover, there are proofs or references to other books, papers,
studies that can provide more detailed information about a certain
topic. Studies, numbers, practices … the first part of the book is
full of them.</p>
<p>The second part is a detailed description of methodology and arguments
why this methods of gathering data are valid. Also nice stuff, for me,
but I think there might be readers bored by this kind of
information. Although I think this knowledge about statistic is also
written in an easy-reading
format.</p>
<p>The third part is a guide on how to perform DevOps transformation in
your organisation, but not very detailed. It is not recipe or step by
step instruction how to be successful on software delivery. You need
to figure it out by yourself, at least all techniques and metrics were
described in the first part of the book.</p>
<p>Overall I recommend <a href="https://amzn.to/3bT4F01">this book</a> to anyone interested in DevOps, high
performing teams and agility. It may be a starting point for many
interesting lectures referenced in. Happy reading!</p>First of all lets write the full title Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations. What nice book it is! I took this book to train while commuting between Gdańsk and Cracow, together with my pal Tomek, The Agile Guy, some time ago and this book was a superb travel companion.Fixing bugs like a pro2018-11-06T00:00:00+01:002018-11-06T00:00:00+01:00https://mergujmyniktniewola.it/2018/11/06/fixing-bugs-like-a-pro<p>Bugs in software are quite natural in my opinion. Probably every developer had
an opportunity to receive a bug report from a client or QA engineer. Some of
such bugs are quite trivial and easy to fix, but some of them are real brain
crackers. The simple ones often do not require some deep understanding, the
solution is easy to introduce. For complex and hard developer needs to dig deep
into the code to understand the error nature. In bug fixing, there are two
things that need to happen: bug needs to be fixed, obviously, and developer need
to prove that this bug will not happen again. But how to make sure both of these
are true? Below I will make a list of how I am resolving bugs.</p>
<h3 id="prove-that-bug-is-real">Prove that bug is real</h3>
<p>How to prove a bug? It is easy - write a test. Sometimes it will be some simple
unit test, sometimes some more complex integration scenario. In most cases, QA
engineer will provide detailed information on how to reproduce bug and
conditions that need to be met. I always make a commit including such a failing
test.</p>
<h3 id="understand-why-bug-happen">Understand why bug happen</h3>
<p>Without understanding the problem introducing a proper fix for complex problems
is impossible. Often writing a test will lead to understanding the nature of a
problem.</p>
<h3 id="introduce-solution">Introduce solution</h3>
<p>Introduce solution and describe it in a commit message. Please do not take an
easy path and confine on simple <code class="highlighter-rouge">Fix</code> message. Probably you won’t be happy
seeing <code class="highlighter-rouge">Fix</code> in <code class="highlighter-rouge">git blame</code> while reading the source code of application you
maintain, right?</p>
<p>So my bugfix pull requests include at least two code commits.</p>
<p><img src="/images/fix-bugs-like-a-pro-commits.png" alt="Commits" /></p>
<p>First is marked with the red cross because of failing specs - bug
proof. Second is bugfix itself. Following commits are optional
refactorings. In my opinion, this is the best approach to fixing bugs
in code.</p>Bugs in software are quite natural in my opinion. Probably every developer had an opportunity to receive a bug report from a client or QA engineer. Some of such bugs are quite trivial and easy to fix, but some of them are real brain crackers. The simple ones often do not require some deep understanding, the solution is easy to introduce. For complex and hard developer needs to dig deep into the code to understand the error nature. In bug fixing, there are two things that need to happen: bug needs to be fixed, obviously, and developer need to prove that this bug will not happen again. But how to make sure both of these are true? Below I will make a list of how I am resolving bugs.The Phoenix Project book review2018-08-06T08:00:00+02:002018-08-06T08:00:00+02:00https://mergujmyniktniewola.it/2018/08/06/the-phoenix-project-book-review<p>After reading <a href="http://mergujmyniktniewola.it/2017/12/09/devops-handbook.html">The DevOps
Handbook</a>
couple of months ago <a href="https://amzn.to/2JHnrvq">Phoenix Project: A Novel about IT, Devops, and
Helping Your Business Win</a> (yes, this is the full title) is the second
book from Gene Kim. It is so much different than the previous one!</p>
<p>I have never read IT book that was written in novel format, with a
narrator and dialogues. Of course, dialogues were not so nice and well
written as in the normal novel but anyway action sometimes was rapid
and addictive. Book has its worse moments, but to be honest Gene Kim
is not a professional action book author. Besides of those short “slow
action” moments after finishing a book, I am curious what will happen
next to Parts Unlimited agile gang. There is no single line of code or
any other pure technical sentences, which is good in my opinion. If
you are looking for a book where you can dive into technical details
of continuous delivery or infrastructure as a code you will not find it
in <a href="https://amzn.to/2JHnrvq">Phoenix Project</a> book. The book describes a story where
understanding principles lead to adopting techniques which elevated
jeopardized business. Without understanding those principles adopting
DevOps techniques may be just following the fashion trend in the
industry. For whom I recommend <a href="https://amzn.to/2JHnrvq">Phoenix Project …</a>? In my opinion,
this is the entry level book for anyone looking into DevOps and
high-performance teams with curiosity but does not know much about it
and had never tried lean practices.</p>
<h3 id="notes">Notes</h3>
<p>Below are a couple of sentences I marked during reading</p>
<blockquote>
<p>The only thing more dangerous than a developer is a developer
conspiring with Security.</p>
</blockquote>
<p>Achieving “over-security” can be dangerous for business. Who needs
security when business is down?</p>
<blockquote>
<p>You will never hit the target you are aiming at if you can fire the
cannon only once every nine months. Stop thinking about Civil Era
cannons. Think antiaircraft guns.</p>
</blockquote>
<p>This is about deployment frequency, I like very much this metaphor.</p>
<blockquote>
<p>One the manufacturing floor, whenever we see work go backward,
that’s rework. When that happens, you can bet the amount of
documentation and information flow is going to be pretty poor, which
means nothing is reproducible and that it’s going to get worse over
time as we try to go faster. The call it ‘non-value-add’ activity or
‘waste’.</p>
</blockquote>
<p>I like so much ‘non-value-add’ activity term.</p>
<blockquote>
<p>DevOps isn’t about automation, just astronomy isn’t about telescopes.</p>
</blockquote>
<p>Do I need to add anything?</p>After reading The DevOps Handbook couple of months ago Phoenix Project: A Novel about IT, Devops, and Helping Your Business Win (yes, this is the full title) is the second book from Gene Kim. It is so much different than the previous one!