<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Tech Lead Tales]]></title><description><![CDATA[Welcome to “Tech Lead Tales” blog. The posts that you’ll find in here will cover the topics that I find critical to make software development projects successfu]]></description><link>https://tltales.com</link><image><url>https://cdn.hashnode.com/res/hashnode/image/upload/v1710794893834/AB2OLXwpZ.png</url><title>Tech Lead Tales</title><link>https://tltales.com</link></image><generator>RSS for Node</generator><lastBuildDate>Thu, 16 Apr 2026 12:31:17 GMT</lastBuildDate><atom:link href="https://tltales.com/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Becoming Tech Lead again]]></title><description><![CDATA[It’s been a while since my last post. During this time, I started working for another company as a regular software developer, not as a Tech Lead. Does that mean I need to close this blog and start a new one with a more fitting title? Just kidding, o...]]></description><link>https://tltales.com/becoming-tech-lead-again</link><guid isPermaLink="true">https://tltales.com/becoming-tech-lead-again</guid><category><![CDATA[tech leadership]]></category><category><![CDATA[onboarding]]></category><dc:creator><![CDATA[Artur Stępniak]]></dc:creator><pubDate>Wed, 18 Sep 2024 21:12:23 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1726693679869/d563b7b5-cef8-49d2-8095-f428bd48c60c.webp" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>It’s been a while since my last post. During this time, I started working for another company as a <strong>regular software developer</strong>, not as a <strong>Tech Lead</strong>. Does that mean I need to close this blog and start a new one with a more fitting title? Just kidding, of course.</p>
<p>My goal is to <strong>become a Tech Lead</strong> again in my current team, and I believe that <strong>writing about the Tech Lead journey</strong> will be a good way to reflect on my progress. I also hope it will be <strong>educational and insightful for developers</strong> like you.</p>
<p>At the time of writing, I've been with this company for about 2.5 months, but this post summarizes <strong>my first month working towards becoming a Tech Lead</strong>.</p>
<h2 id="heading-why-do-i-want-to-be-a-tech-lead">Why do I want to be a Tech Lead?</h2>
<p>This section explains my <strong>motivation for becoming a Tech Lead</strong> and highlights factors you could consider when planning your next career move in software development.</p>
<p>My <strong>motivation to become a Tech Lead</strong> stems from a combination of four key areas:</p>
<ol>
<li><p><strong>What I enjoy</strong></p>
<p> Feeling impactful and responsible for both the <strong>technical and people-management aspects</strong> of projects</p>
</li>
<li><p><strong>What I excel at</strong><br /> Recognizing patterns, analyzing problems, solving complex issues, and optimizing <strong>team workflows</strong> in software projects.</p>
</li>
<li><p><strong>What I want to challenge myself in</strong><br /> Selecting solutions that align with <strong>business goals</strong> and making <strong>architecture-level decisions</strong> that improve long-term scalability.</p>
</li>
<li><p><strong>Compensation</strong><br /> Typically, the more impactful a role is, the better the pay. More importantly, I believe I bring significant value to the companies I work with, and I want to be fairly compensated for that.</p>
</li>
</ol>
<p><strong>Note on #3:</strong> I purposefully titled this point "What I want to <strong>challenge</strong> myself in" rather than "What I want to <strong>develop</strong>" because challenges excite me more. This distinction is important when I revisit this post for <strong>career motivation</strong> or when making future decisions.</p>
<h2 id="heading-my-plan-to-become-a-tech-lead">My Plan to Become a Tech Lead</h2>
<p>In my previous job, I became a <strong>Tech Lead</strong> because I was the most experienced developer, with strong domain knowledge and expertise in software engineering. However, that’s not the case in my current team. This project is also far more demanding in terms of <strong>services performance</strong> compared to my previous projects.</p>
<p>This presents a new challenge for me, especially since I’d like to assume the <strong>Tech Lead role</strong> in a relatively short time. So, what's my plan?</p>
<h3 id="heading-act-like-a-tech-lead-from-day-one">Act Like a Tech Lead from Day One</h3>
<p>I want to show that I have the skills and mindset required to be a <strong>Tech Lead</strong>. I also want my teammates to see what they can expect from me, both as a developer and a potential <strong>team leader</strong> (though I’m not sure they know about my ambition). This is stressful because it involves suggesting changes to people with far greater project knowledge and ingrained work habits. What helps me manage this stress is:</p>
<ul>
<li><p><strong>Determination</strong> to reach my goal</p>
</li>
<li><p>Confidence in my <strong>software engineering skills</strong> and belief that I can propose valuable ideas</p>
</li>
<li><p>A history of failed attempts that have made these situations less intimidating</p>
</li>
<li><p>Acknowledging that I might be wrong, which helps me communicate humbly and listen carefully</p>
</li>
</ul>
<p>But what value can I bring from day one, when I have minimal domain knowledge? Fortunately, <strong>programming languages</strong>, solid <strong>engineering practices</strong>, and <strong>design patterns</strong> are fairly universal. So, I can review code and suggest improvements based on my general expertise.</p>
<p>That’s what I started with, and I received positive feedback for “suggesting a good improvement and bringing fresh energy to the project.” A strong start, I'd say! Code reviews are also a great way to understand team coding practices and what matters most during reviews.</p>
<p>(I was a bit nervous that someone might tell me to ease up on the comments, but no one did. Either my teammates are very patient, or my suggestions were actually valuable.)</p>
<h3 id="heading-listen-observe-identify-problems-and-plan-solutions">Listen, Observe, Identify Problems, and Plan Solutions</h3>
<p>My main goal during the first month was to <strong>observe</strong> the way of working, types of tasks, team dynamics, meet people, and understand the <strong>project’s challenges</strong>.</p>
<p>Meetings with teammates, the architect, and project managers didn’t provide as much insight as I had hoped. However, I learned about two key challenges:</p>
<ol>
<li><p>Many <strong>bugs were being reported</strong> by users.</p>
</li>
<li><p>The team lacked initiative, heavily relying on the Tech Lead, who had been moved to another area. (Spoiler alert - that has improved really fast without anyone’s intervention)</p>
</li>
</ol>
<p>In hindsight, it may have been too early to expect major findings. My ability to dig into issues relies on connecting dots, and I didn’t have enough knowledge at that point. However, the positive side is that I broke the ice with my teammates and other colleagues in the company.</p>
<h3 id="heading-learning-the-domain">Learning the Domain</h3>
<p><strong>Domain knowledge</strong> is crucial for speeding up <strong>feature implementation</strong> and <strong>problem-solving</strong>, so it’s an area I wanted to focus on. However, I must admit that I didn’t make as much progress as I had hoped.</p>
<p>I expected more thorough onboarding into the domain and services my team maintains, but that didn’t happen. In hindsight, I should have requested more detailed onboarding or taken proactive steps to learn on my own.</p>
<p>What I did manage to do was actively participate in <strong>feature refinement</strong> sessions. My approach in these meetings was to act as a full and equal team member, asking for explanations when I didn’t understand something. This might sound bold for someone new to the team, but I believed I could contribute once I understood the project better.</p>
<p>My questions and paraphrasing revealed that not everyone shared the same understanding of certain problems, and some key details were missed. So, I did manage to add value, even at this early stage.</p>
<p>I encourage anyone onboarding to a new project to adopt the mindset of “I deserve to understand,” as it’s a great way to gain domain knowledge. Many people are too afraid to ask for clarifications, but it’s essential.</p>
<h2 id="heading-areas-for-improvement-and-future-plans">Areas for Improvement and Future Plans</h2>
<p>As I mentioned earlier, <strong>domain knowledge</strong> is essential, and I’m not satisfied with my progress in this area. Therefore, it will be my top priority over the coming months. In my previous projects, solving bugs proved to be one of the best ways to gain this knowledge, so that’s how I plan to achieve this goal.</p>
<p>Despite my initial disappointment with bug analysis, I plan to continue efforts to reduce the number of <strong>bugs in our project</strong>. I’ll organize a brainstorming session to explore strategies for achieving this goal.</p>
]]></content:encoded></item><item><title><![CDATA[Case for Modular Monolith]]></title><description><![CDATA[Modular monoliths have fortunately become more popular recently, with many renowned architects and engineers recommending them as the default application architecture. Notably, Sam Newman, a microservices guru, advocates for their use, prompting reco...]]></description><link>https://tltales.com/case-for-modular-monolith</link><guid isPermaLink="true">https://tltales.com/case-for-modular-monolith</guid><category><![CDATA[architecture]]></category><category><![CDATA[monolithic architecture]]></category><category><![CDATA[monolith]]></category><dc:creator><![CDATA[Artur Stępniak]]></dc:creator><pubDate>Tue, 04 Jun 2024 10:01:13 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1717495159725/b70406ae-1425-443d-b81b-98047323fe08.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Modular monoliths have fortunately become more popular recently, with many renowned architects and engineers recommending them as the default application architecture. Notably, Sam Newman, a microservices guru, advocates for their use, prompting reconsideration before starting a new project with a distributed architecture.</p>
<p>This article explores when to use a modular monolith and key considerations for designing one. It also serves as a "teaser" for a project I've been working on for a few months - a modular monolith template/starter written in Python.</p>
<h1 id="heading-when-modular-monolith-is-valid">When modular monolith is valid</h1>
<p>Choosing the right architecture can be challenging, especially when one option becomes the season's buzzword, and the internet claims it's the only valid choice. A modular monolith isn't a silver bullet either. Let's examine its benefits, when it suits a project, and when other architectures should be considered.</p>
<h2 id="heading-benefits-of-modular-monolith">Benefits of modular monolith</h2>
<h3 id="heading-clear-and-fast-features-implementation">Clear and fast features implementation</h3>
<p>The main characteristic of a modular monolith, as the name suggests, is its independent modules (I'll also call them "components" in this post). A key benefit of their autonomy is the speed of introducing new features. There are no complicated, confusing dependencies between different components, making it easier to understand what needs to be done and implement changes quickly.</p>
<h3 id="heading-plays-nicely-with-domain-driven-design">Plays nicely with Domain Driven Design</h3>
<p>Monolith modules correspond to DDD's modules and/or bounded contexts. This means that DDD offers guidance on what modules a given project should have. Both approaches are commonly used in medium to large-sized projects. Coincidence? I don't think so.</p>
<p>Why is this important? Designing system components such as modules, bounded contexts, or aggregates is always challenging. Teams using the DDD approach often employ techniques like Event Storming or User Story Mapping. The outcomes of these techniques help determine how to split the system into modules.</p>
<h3 id="heading-architecture-prepared-for-migration-to-microservices">Architecture prepared for migration to microservices</h3>
<p>Since the modules of a monolith create transactional and semantic code boundaries, they are natural candidates to become separate services. By having modules communicate with one another via clearly defined interfaces and committing changes within a single module, the monolith operates similarly to microservices. Implementing a monolith in this way ensures that some of the necessary mechanisms for a microservices architecture are already in place.</p>
<h2 id="heading-why-not-simply-go-with-microservices">Why not simply go with microservices?</h2>
<p>I've become a big fan of "dull solutions" because they are generally more reliable, easier to maintain, and faster to implement - qualities that businesses typically expect. A modular monolith is one such "dull solution" that, when implemented correctly, also ensures the system can be easily upgraded with new features.</p>
<h3 id="heading-ease-of-introducing-changes">Ease of introducing changes</h3>
<p>Modular monolith is best suited for new or refactored projects with a non-trivial domain.</p>
<p>When starting a new project, you know the least about the domain and business context, so it's rarely wise to implement the system with architectures that make changing component boundaries difficult. In distributed systems, synchronizing changes between multiple services is necessary to maintain proper system functionality. In a modular monolith, you can change boundaries in a single commit.</p>
<p>Distributed systems also introduce added complexity in areas such as observability, security, and communication between components. Configuring these tools and debugging related issues can be time-consuming and hard to justify to the business, whereas these challenges are minimal in a monolith architecture.</p>
<h3 id="heading-non-scalability-myth">Non-scalability myth</h3>
<p>Microservices, lambdas, and other distributed system architectures have become almost synonymous with "scalable architectures," leading some to believe that monoliths cannot be used for performant, replicated services. This is, of course, not true. You can scale a monolith because it is a service like any other. You can even configure some replicas to handle specific high-traffic requests and create more replicas of those "services."</p>
<h3 id="heading-but-it-doesnt-fit-everywhere">But it doesn't fit everywhere</h3>
<p>Of course, a monolith is not a silver bullet. At some point, other architectures may be more suitable. I would opt for microservices when:</p>
<ul>
<li><p>There are so many changes implemented in parallel that testing them takes a lot of time and delays releases. This likely indicates significant organizational growth since the modular monolith was introduced, making it easier, cheaper, and better to change the architecture rather than continuously addressing bottlenecks in the release process.</p>
</li>
<li><p>Some parts of the monolith have very different non-functional requirements, such as scalability, security, or other "-ities." In such cases, it may be better to extract those modules as separate services and use appropriate tools, designs, and infrastructure to meet the specific requirements.</p>
</li>
</ul>
<h1 id="heading-characteristics-of-modular-monolith">Characteristics of modular monolith</h1>
<h2 id="heading-module-responsibility">Module responsibility</h2>
<p>In the Clean Code approach, every class and function has a clearly defined responsibility. The same applies to modules, though deciding on their boundaries is not an easy task. Do it wrong, and you'll end up with coupled code, making it difficult to implement new functionalities or fixes without changing multiple modules. Do it right, and you'll benefit from fast changes, easier testing, and maintainable code.</p>
<p>What is right and what's not? Let's check with an example of a system that creates simple games for kids and adults. Each game consists of tasks to accomplish, a background story, and instructions. The story and instructions have both text and speech forms.</p>
<p>Initially, the team implemented this with the following modules: adult game, game instructions, and kids game. While some dependencies between the modules are inevitable, the most problematic one is the "generate speech" function defined in the "kids game" module but used by all modules. This can make the function hard to maintain.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1716712284482/5f1964f9-2ad1-423b-8538-399958a25107.jpeg" alt class="image--center mx-auto" /></p>
<p>After refactoring, this might look like the following:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1717273272865/0f56a896-4fe6-41e0-aacc-9643698d20ef.jpeg" alt class="image--center mx-auto" /></p>
<p>In the case of very simple games, it may be possible to implement the <code>game</code> module's logic with generic code that is parameterized by the player's type. This approach eliminates the need for a separate <code>game instructions</code> module. However, it's still beneficial to separate the text-to-speech functionality into its own module to facilitate easy use by other modules. This way, the speech generation logic remains encapsulated and reusable across different parts of the system.</p>
<h2 id="heading-module-interface">Module interface</h2>
<p>No matter how hard you try, communication between modules becomes inevitable when implementing a system that involves more than CRUD operations. In the example of the Games System, the <code>game</code> module utilizes <code>text-to-speech</code> functionality.</p>
<p>It's crucial for downstream modules to expose interfaces that hide their internal implementation details, allowing upstream modules to only use those endpoints. This decoupling ensures that a calling component does not need adjustment when changes are introduced in a called component.</p>
<p>The choice of interfaces depends on your approach. These may include services, controllers, commands, events, etc. The key is to commit to using only these interfaces for communication between modules, promoting modularization and maintainability.</p>
<h2 id="heading-transactional-boundary">Transactional boundary</h2>
<p>Deciding whether a module should define its transactional boundary requires careful analysis, as it can introduce complexities into your system. If each module commits only the data it manages, then the system:</p>
<ul>
<li><p><strong>Is more error-prone</strong>, as upstream modules still commit their changes even if downstream processing is buggy.</p>
</li>
<li><p><strong>Has better performance</strong>, as downstream request handling may occur in the background.</p>
</li>
</ul>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1717274930491/c5acdd03-f820-4001-a33c-d72f79cec2ce.png" alt="processing request succeeds right after handler A commits changes. Failure of handler B does not cause request processing failure and long processing of handler C does not impact processing time of the request" class="image--center mx-auto" /></p>
<ul>
<li>Combines the above two characteristics: fewer entities are committed in a single transaction, increasing the probability of conflict-free transactions and eliminating the need for retries or other handling mechanisms to process the request.</li>
</ul>
<p>To benefit from the mentioned traits, you need to make communication between modules more intricate:</p>
<ol>
<li><p><strong>Ensuring System Consistency:</strong> Addressing system consistency in case of downstream errors is crucial. While upstream processing may succeed even if downstream modules fail, it leaves the system in an inconsistent state that needs handling. Implementing a proper mechanism is essential. This could involve retrying processing, fixing bugs, or finding alternative solutions.</p>
</li>
<li><p><strong>Handling "Only Once Processing":</strong> Dealing with the issue of "only once processing" is vital. Suppose a request is processed by a handler that triggers multiple downstream processors, some of which succeed while others fail. To address this:</p>
<ul>
<li><p>Ensure all handlers are idempotent to safely retry downstream operations. While it's often feasible to make a processor idempotent, assuming all can be safely retried poses risks.</p>
</li>
<li><p>Retry only the handlers that failed. This is a non-trivial problem but can be addressed using patterns such as the inbox and outbox patterns. These patterns help manage the state and retry logic for failed operations, ensuring that only the necessary handlers are retried.</p>
</li>
</ul>
</li>
</ol>
<p>As you can see, module-bounded transactions can provide profound benefits but at the cost of more complex system. I'd say it's important to discuss the value of the advantages with business people to decide if it's worth complicating the design and maintenance.</p>
<p>It's essential to remember that the decision to separate transactions isn't all-or-nothing. You can isolate transactions in product-critical modules while sharing them in other components. However, transitioning from a shared transaction to separate ones can be challenging, especially if your code lacks clearly defined interfaces.</p>
<p>In the upcoming posts, I'll detail how I tackled various challenges in my hobby modular monolith project. Stay tuned, and feel free to post any comments or questions until then!</p>
]]></content:encoded></item><item><title><![CDATA[Hazards of backend async-ronization in Python]]></title><description><![CDATA[Asynchronous code is great if you want to get the most out of your backend system that also sends requests to external systems or does other I/O operations. Refactoring synchronous backend code to get those benefits is not that great, or at least not...]]></description><link>https://tltales.com/hazards-of-backend-async-ronization-in-python</link><guid isPermaLink="true">https://tltales.com/hazards-of-backend-async-ronization-in-python</guid><category><![CDATA[Python]]></category><category><![CDATA[asynchronous]]></category><category><![CDATA[async]]></category><category><![CDATA[FastAPI]]></category><category><![CDATA[Gunicorn]]></category><category><![CDATA[uvicorn]]></category><dc:creator><![CDATA[Artur Stępniak]]></dc:creator><pubDate>Wed, 20 Mar 2024 12:00:21 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1710875622143/09705453-fd44-4ff2-b7bb-da504bcf2ad5.webp" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Asynchronous code is great if you want to get the most out of your backend system that also sends requests to external systems or does other I/O operations. Refactoring synchronous backend code to get those benefits is not that great, or at least not as easy as I had anticipated.</p>
<p>The post turned out quite lengthy (again) so here’s what it’s about:</p>
<ol>
<li><p>Crashing Gunicorn worker - client receives an empty response when blocking I/O operation is executed</p>
</li>
<li><p>Initialization of <code>ContextVar</code> in global context - client’s data accessible to other clients</p>
</li>
</ol>
<h1 id="heading-crashing-gunicorn-worker">Crashing Gunicorn worker</h1>
<p>In the case of my project we were not that much on a hunt for performance gains but wanted to benefit from automatic documentation that FastAPI provides. However, we defined our middlewares and endpoints <code>async</code> but left most of our code synchronous. And that peculiar mix resulted in crashes of Gunicorn workers.</p>
<h3 id="heading-minimum-reproducible-example">Minimum reproducible example</h3>
<p>Here’s a simplified version of our code</p>
<pre><code class="lang-python"><span class="hljs-keyword">import</span> time
<span class="hljs-keyword">from</span> fastapi <span class="hljs-keyword">import</span> FastAPI

app = FastAPI()

<span class="hljs-meta">@app.get("/")</span>
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">read_root</span>():</span>
   start = time.time()
   <span class="hljs-comment"># this can be a request sent to another service</span>
   <span class="hljs-comment"># or any other I/O operation</span>
   time.sleep(<span class="hljs-number">30</span>)
   <span class="hljs-keyword">return</span> {<span class="hljs-string">f"This took <span class="hljs-subst">{time.time() - start}</span> seconds to complete"</span>}
</code></pre>
<p>What’s important is that there's a blocking operation when handling requests - <code>time.sleep</code> instead of <code>await asyncio.sleep</code>.</p>
<p>When the above application is run via a Gunicorn:</p>
<p><code>gunicorn -k uvicorn.workers.UvicornWorker 'main:app' -b localhost:8000 -t 30</code></p>
<p>sending a request results in a quite cryptic error</p>
<pre><code class="lang-bash">curl localhost:8000
curl: (52) Empty reply from server
</code></pre>
<p>which becomes apparent after checking server logs</p>
<pre><code class="lang-bash">[2024-02-27 21:57:53 +0100] [18085] [CRITICAL] WORKER TIMEOUT (pid:18110)
[2024-02-27 21:57:54 +0100] [18085] [ERROR] Worker (pid:18110) was sent code 134!
</code></pre>
<p>As you probably guess, that’s caused by the blocking sleep in handling the request which purely by accident is the exact value of Gunicorn’s worker timeout. That is clear in case of such a simple code snippet but may be harder to debug in a production code. In our case handling a request involved exchanging quite an amount of requests with an external service (which is notorious for causing integration problems).</p>
<h3 id="heading-what-is-exactly-going-on">What is exactly going on?</h3>
<p>The main process of Gunicorn essentially does 2 things:</p>
<ol>
<li><p>Forwards requests to it’s workers</p>
</li>
<li><p>Manages workers - starts them up, kills if they are not responsive for a specific length of time (yes, the timeout that is passed to <code>gunicorn</code> command), etc.</p>
</li>
</ol>
<p>OK, but the worker is actually alive so why such harsh treatment?</p>
<p>It’s alive but not able to respond to “poking” done by Gunicorn’s main process. Because of the blocking operation the execution control is not handed back to Uvicorn event loop.</p>
<p>Here’s an illustration of what would happen if <code>await asyncio.sleep</code> was used</p>
<p><img src="https://lh7-us.googleusercontent.com/uj4N5om2qXujroFaofgTfenRAiW6Ke1JG0drLOc-NkQNvNenA3o-a8NBJ1kaVQYh8AvL8X7C2hD5YslxTbVpaApOQbwTdnd0-A2OUejI12bpZLw3F_F-QpSmryvhofDy_x73-bkKA-hmqCXx690RK_k" alt /></p>
<p>In that case the execution would return to the event loop which would then call a function that handles the “poke”.</p>
<p>Let’s use the non-blocking operation then:</p>
<pre><code class="lang-python"><span class="hljs-keyword">import</span> asyncio
<span class="hljs-keyword">import</span> time

<span class="hljs-keyword">from</span> fastapi <span class="hljs-keyword">import</span> FastAPI

app = FastAPI()

<span class="hljs-meta">@app.get("/")</span>
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">read_root</span>():</span>
   start = time.time()
   <span class="hljs-comment"># this can be a request sent to another service</span>
   <span class="hljs-comment"># or any other I/O operation</span>
   <span class="hljs-keyword">await</span> asyncio.sleep(<span class="hljs-number">30</span>)
   <span class="hljs-keyword">return</span> {<span class="hljs-string">f"This took <span class="hljs-subst">{time.time() - start}</span> seconds to complete"</span>}
</code></pre>
<p>The response is then</p>
<pre><code class="lang-bash">curl localhost:8000
[<span class="hljs-string">"This took 30.002015352249146 seconds to complete"</span>]%
</code></pre>
<p>and no sign of problems in the server logs</p>
<pre><code class="lang-bash">gunicorn -k uvicorn.workers.UvicornWorker <span class="hljs-string">'main:app'</span> -b localhost:8000 -t 30
[2024-02-27 22:40:32 +0100] [21686] [INFO] Starting gunicorn 21.2.0
[2024-02-27 22:40:32 +0100] [21686] [INFO] Listening at: http://127.0.0.1:8000 (21686)
[2024-02-27 22:40:32 +0100] [21686] [INFO] Using worker: uvicorn.workers.UvicornWorker
[2024-02-27 22:40:32 +0100] [21710] [INFO] Booting worker with pid: 21710
[2024-02-27 22:40:33 +0100] [21710] [INFO] Started server process [21710]
[2024-02-27 22:40:33 +0100] [21710] [INFO] Waiting <span class="hljs-keyword">for</span> application startup.
[2024-02-27 22:40:33 +0100] [21710] [INFO] Application startup complete.
[2024-02-27 22:41:53 +0100] [21686] [INFO] Handling signal: winch
</code></pre>
<h3 id="heading-solution-bigger-and-smaller-workarounds">Solution, bigger and smaller workarounds</h3>
<p>What can be done to avoid such problem:</p>
<ol>
<li><p>Employ non-blocking operations. This task is challenging with legacy code. Every function leading to the troublesome blockage requires modification, including decorators. This issue intensifies if the altered code must remain compatible with synchronous operations.</p>
</li>
<li><p>Run the synchronous code in a threadpool. The below code does not block</p>
</li>
</ol>
<pre><code class="lang-python"><span class="hljs-keyword">import</span> time

<span class="hljs-keyword">import</span> anyio.to_thread
<span class="hljs-keyword">from</span> fastapi <span class="hljs-keyword">import</span> FastAPI
<span class="hljs-keyword">from</span> starlette.concurrency <span class="hljs-keyword">import</span> run_in_threadpool

app = FastAPI()

<span class="hljs-meta">@app.get("/")</span>
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">read_root</span>():</span>
   start = time.time()
   <span class="hljs-comment"># this can be a request sent to another service</span>
   <span class="hljs-comment"># or any other I/O operation</span>
   <span class="hljs-keyword">await</span> anyio.to_thread.run_sync(time.sleep, <span class="hljs-number">30</span>)
   <span class="hljs-keyword">return</span> {<span class="hljs-string">f"This took <span class="hljs-subst">{time.time() - start}</span> seconds to complete"</span>}
</code></pre>
<p>This is a great option and Starlette/FastAPI actually uses that “trick” to call synchronous middleware or endpoint. Since web frameworks do it automatically, then it’s also a valid option to give up on asynchronous endpoints if your use case allows it.</p>
<ol start="3">
<li>Get rid of Gunicorn. That may sound controversial but that’s actually a valid option if your app is orchestrated by Kubernetes. In that case Kubernetes manages the “workers” (pods) and Gunicorn doesn’t provide big value.</li>
</ol>
<h1 id="heading-initialization-of-contextvar-in-global-context">Initialization of <code>ContextVar</code> in global context</h1>
<p>That one was really hard to debug and was caused in big part by how the dependency injection library we use is implemented but it can occur also in case of the code you write yourself. I’ll make the case easier to understand.</p>
<h3 id="heading-intro-contextvar-in-async-python">Intro: ContextVar in async Python</h3>
<p>When processing a request quite often you need to store a state in memory so that it is accessible later in the chain of processing. This could be for example user information (id, email, roles etc.) determined in a middleware based on authentication token and referenced in application or domain code.</p>
<p>Asynchronous code runs, by default, in a single process and thread so if you were to store this data in a global variable, it’d be accessible, or even overwritten by other requests processing.</p>
<p><code>ContextVar</code> is a solution for such case. It serves the same goal as <code>threading.local</code> and can replace it. The value of a <code>ContextVar</code> will be different for each “chain” of asynchronous processing code</p>
<pre><code class="lang-python"><span class="hljs-keyword">import</span> asyncio
<span class="hljs-keyword">import</span> contextvars

email: contextvars.ContextVar[str] = contextvars.ContextVar(<span class="hljs-string">"email"</span>)


<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_email</span>() -&gt; str:</span>
   <span class="hljs-keyword">return</span> email.get()


<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">set_email</span>(<span class="hljs-params">value: str</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
   print(<span class="hljs-string">f"Setting email to <span class="hljs-subst">{value}</span>"</span>)
   email.set(value)


<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">email_processing</span>(<span class="hljs-params">value: str, seconds_of_sleep: int</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
   <span class="hljs-keyword">await</span> set_email(value)
   <span class="hljs-keyword">await</span> asyncio.sleep(seconds_of_sleep)
   print(<span class="hljs-string">f"<span class="hljs-subst">{<span class="hljs-keyword">await</span> get_email()}</span>, expected <span class="hljs-subst">{value}</span>"</span>)


<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">main</span>() -&gt; <span class="hljs-keyword">None</span>:</span>
   <span class="hljs-keyword">await</span> asyncio.gather(
       email_processing(<span class="hljs-string">"value1@example.com"</span>, <span class="hljs-number">1</span>),
       email_processing(<span class="hljs-string">"value2@example.com"</span>, <span class="hljs-number">0</span>),
   )

<span class="hljs-keyword">if</span> name == <span class="hljs-string">"__main__"</span>:
   asyncio.run(main())
</code></pre>
<p>This prints</p>
<pre><code class="lang-bash">python contextvar_example.py  

Setting email to value1@example.com
Setting email to value2@example.com
value2@example.com, expected value2@example.com
value1@example.com, expected value1@example.com
</code></pre>
<p>Using <code>ContextVar</code> is quite easy but, as you’ll see below, it’s still possible to introduce bugs 🙂</p>
<details><summary>State provided by web frameworks</summary><div data-type="detailsContent">By the way, if you don’t need a “clean code” solution for managing state in asynchronous backend, then the framework you use probably does it for you and provides you with a mechanism to store and read state. In <a target="_blank" href="https://www.starlette.io/requests/#other-state">FastAPI/Starlette it’s <code>state</code> variable</a></div></details>

<h3 id="heading-global-contextvar">Global ContextVar</h3>
<p>The bug that we implemented was that one of our <code>ContextVar</code> variable sometimes became global - shared between requests - which caused overwriting data and accessing other users data (thankfully we noticed it before deploying the changes to production).</p>
<p>The following simplified example illustrates the problem:</p>
<pre><code class="lang-python"><span class="hljs-keyword">import</span> asyncio
<span class="hljs-keyword">import</span> contextvars

request_state: contextvars.ContextVar[dict] = contextvars.ContextVar(
    <span class="hljs-string">"email"</span>
)


<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">app_startup</span>() -&gt; <span class="hljs-keyword">None</span>:</span>
   request_state.set({})


<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">process_request</span>(<span class="hljs-params">email: str, processing_time: int</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
   state = request_state.get()
   state[<span class="hljs-string">"email"</span>] = email
   <span class="hljs-keyword">await</span> asyncio.sleep(processing_time)
   print(<span class="hljs-string">f"Processing request with <span class="hljs-subst">{state=}</span>, expected email <span class="hljs-subst">{email}</span>"</span>)

<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">main</span>() -&gt; <span class="hljs-keyword">None</span>:</span>
   <span class="hljs-keyword">await</span> app_startup()
   <span class="hljs-keyword">await</span> asyncio.gather(process_request(<span class="hljs-string">"value1@example.com"</span>, <span class="hljs-number">2</span>), process_request(<span class="hljs-string">"value2@example.com"</span>, <span class="hljs-number">1</span>))

<span class="hljs-keyword">if</span> name == <span class="hljs-string">"__main__"</span>:
   asyncio.run(main())
</code></pre>
<p>which results in</p>
<pre><code class="lang-bash">python global_context_var.py

Processing request with state={<span class="hljs-string">'email'</span>: <span class="hljs-string">'value2@example.com'</span>},
    expected email value2@example.com
Processing request with state={<span class="hljs-string">'email'</span>: <span class="hljs-string">'value2@example.com'</span>},
    expected email value1@example.com
</code></pre>
<p><code>ContextVar</code>’s value is set during application startup which makes it shared with subsequent coroutines and/or tasks. When the state is updated during requests processing, it’s also updated for the other coroutine so instead of <a target="_blank" href="mailto:value2@example.com"><code>value1@example.com</code></a> the first request is processed with <code>value2@example.com</code>.</p>
<p>Again, it’s relatively easy to spot the problem in the above exemplary code but most of the time there’s a lot of indirection. In our case there were 2 cases:</p>
<ol>
<li><p>Our dependency injection library uses descriptors which were evaluated during application startup and caused a <code>ContextVar</code> to be initialized (instances of dependencies are stored in the <code>ContextVar</code>)</p>
</li>
<li><p>Due to infrastructure problems an exception during app startup was raised which caused getting an instance of Sentry reporter via dependency injection library… you know the rest.</p>
</li>
</ol>
<h3 id="heading-solution-workarounds">Solution, workarounds</h3>
<p>I can’t think of any pattern that would protect the codebase from introducing such bugs, so sorry, no general solution here. If you are aware of such, please let know in the comments.</p>
<p>We did, however, introduce a safety mechanism - a middleware that checks whether the <code>ContextVar</code> has a value set, reports an error to Sentry if that’s true, resets the variable and proceeds with processing.</p>
<h2 id="heading-request-for-patterns">Request for patterns</h2>
<p>As you can see, even though asynchronous programming concepts in Python are quite simple it’s possible to get into complex problems. If you know some other problematic cases, patterns for solving them or clever ways to refactor code to support <code>async</code>, please write in the comments. Same if you’re interested in some other topics related to <code>async</code> in Python.</p>
]]></content:encoded></item><item><title><![CDATA[Beyond Code: How Engineers Propel Business Growth]]></title><description><![CDATA[Clients expect engineers not only to deliver performant, maintainable code. Best software developers make decisions with business growth in mind. Thanks to that, return on investment can be maximized, costs minimized and engineers can be proud of the...]]></description><link>https://tltales.com/beyond-code-how-engineers-propel-business-growth</link><guid isPermaLink="true">https://tltales.com/beyond-code-how-engineers-propel-business-growth</guid><category><![CDATA[engineering]]></category><category><![CDATA[Business growth ]]></category><category><![CDATA[software development]]></category><category><![CDATA[continuous deployment]]></category><category><![CDATA[team]]></category><dc:creator><![CDATA[Artur Stępniak]]></dc:creator><pubDate>Sun, 17 Mar 2024 14:42:54 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1710686825655/93965c49-438d-40dd-b020-4dbc696948b5.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Clients expect engineers not only to deliver performant, maintainable code. Best software developers make decisions with business growth in mind. Thanks to that, return on investment can be maximized, costs minimized and engineers can be proud of the solutions they deliver.</p>
<p>It's not enough to implement features. In this post you'll find out what engineers should do to help business grow.</p>
<h2 id="heading-discussing-idea-and-gathering-requirements"><strong>Discussing idea and gathering requirements</strong></h2>
<p>Clients should involve engineers in discussions when they have new feature ideas. It’s a stage when it’s easiest to introduce changes and avoid costly implementations.</p>
<h3 id="heading-suggesting-alternative-solution"><strong>Suggesting alternative solution</strong></h3>
<p>Quite often Clients will describe an idea for a solution to a problem they have. However, quite often, the idea is overly complicated.</p>
<p>Validating an idea is usually the role of a Business Analyst. But developers should still be critical of ideas. This is because they have knowledge of technical aspects that may make a suggested solution complicated.</p>
<p>When discussing the idea with clients, engineers and BAs should approach it with curiosity and critical thinking. This will help with uncovering the underlying problem that the feature is intended to solve.</p>
<p>Curiosity, from my experience, is not a problem in IT. It's what brings people here.</p>
<p>Critical thinking, or assertiveness, is something that is gained through experience. That’s why it’s senior developers' role to analyze an idea and say that the problem should be solved in another way.</p>
<p>Uncovering and discussing the core problem is a skill and, like any skill, can be learnt. Starting with asking <a target="_blank" href="https://www.wikiwand.com/en/Open-ended_question">open-ended questions</a> is good, but if the problem or feature is more complex it’s not enough. Business Analysts and developers should be capable of leading the conversation with a client - structuring the discussion and asking the right questions. There’s a reason why conversations with domain experts are so important in Eric Evans’ “Domain Driven Design” book.</p>
<p>If you want to get skilled at discussing problems with clients, I encourage you to read <a target="_blank" href="https://www.michalbartyzel.pl/ksiazki/">a book by Michal Bartyzel</a> “Conversation Patterns for Software Professionals” (or “Oprogramowanie szyte na miarę” if you know Polish; it’s a bit extended version of the English book).</p>
<h3 id="heading-cutting-on-scope"><strong>Cutting on scope</strong></h3>
<p>What’s important to understand is that the client wants to validate the solution and get the major benefits ASAP. After leading the conversation with the client BA and engineers should easily realize what functionality subset will deliver the most benefits in the shortest time.</p>
<p>In my project we often implement the mobile-app facing features but delay or skip developing management of those in CMS. It’s the easiest division for us.</p>
<p>What can be challenging is communicating the scope cuts to the client. It's hard especially when the client insists on prioritizing non-core parts of a feature.</p>
<p>I need to confess that quite recently I failed at challenging the client. Because of that, my team spent a few weeks on changes that the client eventually acknowledged should have had a lower priority. What I missed was clearly showing the benefits the core functionality already delivered.</p>
<h2 id="heading-implementation-analysis-and-planning"><strong>Implementation analysis and planning</strong></h2>
<p>After the client, BA and engineers agree on the scope it’s time for developers to choose the technical solutions and plan the implementation so that the costs and potential risks are minimized and benefits maximized.</p>
<h3 id="heading-presenting-limitations-and-solutions"><strong>Presenting limitations and solutions</strong></h3>
<p>Not long ago we were analyzing an upcoming feature which relied on integration with 3rd party services. We realized that quite big delays may be introduced by those external partners. They were caused by  the nature of their services and the way they were used by users.</p>
<p>What I did was prepare a simple diagram that showed the impact of those limitations on the feature and users. That seems easy but I noticed that developers tend to use technical language when talking with clients. That makes discussions longer and results in clients misunderstanding the message.</p>
<p>Business oriented engineers also propose solutions to minimize their impact. That requires being aware of the goals of the feature to be implemented which brings us back to asking the right questions. Communicating different solutions can be challenging when solutions are quite technical. But in most cases it’s possible to show how different options impact:</p>
<ul>
<li><p>time to implement the feature,</p>
</li>
<li><p>code maintainability (that includes scalability and resiliency),</p>
</li>
<li><p>user experience.</p>
</li>
</ul>
<h2 id="heading-incremental-implementation-and-gradual-deployment"><strong>Incremental implementation and gradual deployment</strong></h2>
<p>It’s the last stage so introducing changes is harder than in the previous ones. Yet, engineers can still use some techniques to maximize business benefits.</p>
<p>There are 2 general tools:</p>
<ul>
<li><p>planning gradual deployment,</p>
</li>
<li><p>incremental implementation,</p>
</li>
</ul>
<p>It’s important to note that the 1st one can make the 2nd much easier to plan and execute.</p>
<h3 id="heading-gradual-deployment"><strong>Gradual deployment</strong></h3>
<p>In the feature my team is working on right now we decided that it’d be good to release it to the users gradually. The advantages of such approach are:</p>
<ul>
<li><p>gathering initial feedback about the solution,</p>
</li>
<li><p>have only part of the users impacted by potential bugs and,</p>
</li>
<li><p>buy us some time to make our code properly scalable (what I like most).</p>
</li>
</ul>
<p>A/B and canary releases are already quite popular which means there are tools that make it easier. Our project uses Firebase for deciding whether a user should see the new feature or not.</p>
<h3 id="heading-incremental-implementation"><strong>Incremental implementation</strong></h3>
<p>In Scrum the common way of working with big features is splitting them into smaller parts and implementing them one by one. That’s good for knowing the actual progress of the functionality development. Engineers should be able to find technical solutions for features to bring benefits ASAP.</p>
<p>This can be very challenging when the requested feature is totally different from all the other ones. Quite often this requires use of new frameworks, tools or architectural approaches. But it doesn’t need to delay bringing benefits of the new functionality.</p>
<p>Engineers should analyze whether the feature can be implemented in stages/incrementally. Initial stage should deliver working functionality that users can already take advantage of. This stage can compromise on resources usage efficiency, scalability or other factors depending on the functionality and technical solutions. While it’s already working the next stages can be developed and delivered.</p>
<p>What we lately developed in my team is fetching data from the external service via a periodic task. Implementing it that way has several drawbacks, such as wasting resources for sending requests that return no new data. But the big advantage is that we were able to deliver it fast. The next step is to create a new service which exposes a webhook for the external service to push data to. We decided to have it as a second stage of the implementation because this approach requires us to solve problems related to distributed programming and new kubernetes deployment setup, which we don’t have experience with.</p>
<p>As you can see, when planning the implementation engineers should take into account not only the technical solutions optimal for the feature but also their skillset and its impact on risks.</p>
<p>There are 2 main challenges with this approach:</p>
<ul>
<li><p>Developers need to make peace with the idea that playing with new, exciting solutions will be delayed,</p>
</li>
<li><p>Clients need to reserve engineers’ capacity for the later stages of implementation. Abandoning subsequent phases delivery poses risks. That requires developers to properly communicate the hazards.</p>
</li>
</ul>
<h2 id="heading-communicate-communicate-communicate"><strong>Communicate, communicate, communicate…</strong></h2>
<p>As you can see engineers have a big role in minimizing costs and maximizing benefits. It’s important for developers to have proper attitude and communication skills. That’s why I encourage you to develop yourself and your team in this area. Pay also attention to what you can improve in the business oriented approach.</p>
]]></content:encoded></item></channel></rss>