The complete software engineer

Technology and automation are tapping into every possible industry and the demand for software engineers has never been higher. Building successful companies gets harder every time because of all the competition creating new challenges everywhere, and as more people try to get themselves a good slice of the giant IT pizza, the need for differentiation between engineers becomes crucial to get a taste of that pepperoni.

The software engineer role

#

But what do exactly SWEs do ? There are many areas of specialisation like embedded systems, AI, Data or Web, all of them different from each other but sharing the same engineering foundations. In the dot com bubble days, IT companies defined clear borders between developers and every other role in the business. A typical setup would include different people for infrastructure, systems, databases, security, technical support, and software development, which was what they knew to be the accepted formula on other industries like construction, electronics or automobile. Around 20 years after, a huge and competitive internet (4,6 billion users - 60 percent of the world population) pushed software systems and the underlying technology to different levels and the SWE role had to evolve in order to keep up with the new reality. While the classic setup is still around in many companies, the borders between roles are blurrier and we see SWEs taking responsibilities on every part of the stack. That’s because our expected deliverables are now much closer to the resulting products and services, rather than “just code”.

Delivering Products and Services

#

Ahead of this paradigm shift, large IT companies started picking up signals from the growing number of successful startups. How did organisations with less people and resources manage to deliver better products within a shorter time to market ? Part of it was because their teams were small and cohesive. When you’re on a tight budget you can’t afford to staff a traditional IT structure, so on one side SWEs at startups would have to build the full technology stack to support the product vertical, and on the other they didn’t suffer from the hassle of team dependencies or communication issues. Cloud service and infrastructure providers also play a key role in this transformation, lifting off the significant amount of work required to build production ready runtimes, making life easier for everybody whose product depends on a database, CDN, message queue, container scheduler etc. Eventually lots of companies like Amazon, Airbnb, Spotify, Valve, etc. iterated over engineering team setups, creating different and innovative patterns that most organisations follow today. Having said all of this, the current state of the art indicates that today’s SWEs are required to work on a scope that goes beyond their code. This could materialise in different forms, but put it simply, companies prioritise people with diverse skills rather than specialised ones.

Differentiating

#

How can we differentiate then ? How do we reinvent ourselves to be more aligned with market demands ? This is one of those vague questions that have no single correct answer, however, having worked in a couple of different industries, I found out that there are some simple vectors we can follow to maximise our value to the company, and ultimately as engineers. Following this set of principles over the last few years helped me to perform better both as a developer and tech lead.

Understand the company and the product

#

I cannot stress how important this is. In the early days of my career, I was avid to get cracking with new programming languages, frameworks, struggling to create better code than my peers, without ever really thinking about what would be real world applications of what we were creating. While It had its limited value, I often ended up disagreeing with specific tasks, delivering features of no use to the customers, or needing to reiterate because the deliverables were not fit for purpose. The IT industry created a psychological barrier between engineers and the outside world and it was actually after breaking that barrier that I began to understand the real value and the cost of the things we do. Companies appreciate engineers that are able to deliver what they want to sell, and minimise the waste of money, which is way easier under those circumstances. The other hidden benefit of understanding what you are doing, is that it will make you feel part of the product and more accountable for it, which ultimately can lead to being more motivated and performing better, or choosing a different career path because you don’t agree with your company’s strategy.

Work collectively, not individually

#

There’s an anti-social, competitive, misfit aura above SWEs (we’re seen as The Big Bang Theory characters by most people) because, well, some of us prefer to be around machines rather than people. That’s also a thing of the past. Nowadays, engineering is of the most culturally diverse and inclusive qualified professions and it has to be because the market is way to big and competitive for one man bands. There are exceptions, the unicorns of course, but the norm for scaling up businesses is get more people, and in almost every case, successful products are the coordinated results of multiple teams. Collective work brings social and cultural challenges that go beyond technical skills. Throughout my career I have felt the frustration of disagreement or others standing in my way, and most likely I caused frustration on them too. Luckily enough, i’ve crossed paths with several wise people who taught me to sit on other people’s shoes before making an opinion. In the end it is the company’s success that is on the line, and one’s success or failure will eventually end up reflected on everybody’s pay check. If you always see things in that perspective It’ll be easier to break what I call the FarmVille mindset (working in silos, unaware of the big picture). Positive attitude towards others delivers immediate value to the company and long term value to yourself. Moreover, it’s wrong to think that It will interfere with your personal agenda, you’ll just have to figure out a different way to carry on with it, and when you do, you’ve become a better engineer.

Document your work

#

Some time ago, at company X, I was doing a one man job for this bespoke project that our product manager asked me to do. The delivery was a success, customer was happy, documentation zero. After a while I wasn’t around anymore and the customer requested some changes. You can figure out the rest right ? Unless you want to stay in the same job forever (not criticising), your presence everywhere will be temporary, so you need to cover that by leaving tracks for the team to follow. Documentation is hard and boring, especially because it doesn’t always deliver short-term results. It is even harder if you start piling up on items to document, so the best option is really to embrace it and consider it a part of your deliverables. As a developer I never really cared much about documentation, and it took me a while to understand that if you always invest some time in making it easier to the next person, there will be less questions and dependencies on yourself in the future. Moreover, it makes your work more visible and exposed to the rest of the company, so in the end it’s not only important for the team and the company’s ability to use it, but it’s also an investment in yourself.

Test every deliverable

#

This is probably one of the most popular discussions in the software engineering space, yet, due to the stressful urgency to deliver and getting to market as soon as possible, companies tend to commit with aggressive timelines and tests are one of the first things that get discarded to match those dates. Consequences are somewhat predictable, so this is one of those no brainers that depend more on culture than skills. I’ve worked in projects on both extremes (no tests vs excessive tests) and as expected, having no tests led to production issues affecting customers more often while excessive tests led to less surprises and bigger, costly development cycles. Not investing time in testing is obviously worse for the company, but the aggressive timelines will always be there, since very few businesses have time and money to waste. It’s the SWEs job to find an effective testing strategy that fits the team’s reality and has the right balance between missing test coverage and extreme testing. With a good testing strategy established and enforced by everyone, changes will be smoother, less prone to errors, increasing product reliability and your credibility as an engineer.

Monitor your systems

#

It’s not really important during development, nor it is on your local machine. But have you ever thought how would it be if there were no fire alarms, airbags or thermostats ? Monitoring is how you know what’s happening in your systems and it’s what enables you to anticipate or mitigate outages. Traditional software development teams didn’t usually have this kind of concerns because the maintenance and operation of the production runtime would sit under another team’s scope, that would care about it. Monitoring isn’t a new thing, it’s been there since computer programs exist, we’re just seeing it more on every team, because the culture is changing and consequently, pretty much every framework or application has a way to export metrics. Wether it’s by using logs, or metrics, or tracing, your team will welcome a good observability setup on the features you deliver, because it will make everybody’s life easier throughout their maintenance.

Design secure systems

#

Not so long ago I was on a mission to deliver a new product. The team had been running against the clock to match the release date and by the time it arrived, the product was visually ready for launch, so we did it. With very little experience on security and privacy I didn’t pay much attention to the APIs being exposed, but after so much effort, clicking that deploy button felt like doing a good thing for the company. But was it really ? Within 72 hours after launching we suffered an enumeration attack and had to mobilise lots of people to fix the breach and pick up the scraps. Rookie mistakes like this can be damaging, even if you’re not building customer facing products, so it’s really one of those concerns that will make you a better SWE. Security is not as popular in discussion as testing, monitoring and documentation, but it’s probably more important due to what can happen when things go wrong. Following standard authentication and encryption methods is a good place to start but they’re not enough. It’s hard to build a secure system, but it’s harder to fix an insecure one and deal with the potential consequences, so don’t keep yourself from bringing the topic to the table when building software with your team.

Take full ownership of your domain

#

Product teams are like mini startups inside a company, responsible for the full product lifecycle from development to maintenance. In order to maximise the team’s effectiveness in doing so, every team member should be well aware of the product’s technical borders and be comfortable operating inside them. The thing is that, as I mentioned several times in this article, those borders will at certain point go beyond your code. In other words, as a modern SWE, you need to be prepared to do more than just software development tasks. Depending on the context this can go from orchestrating infrastructure, databases, attending meetings with relevant stakeholders, leveraging other teams that the product depends on, etc. This doesn’t mean that SWEs should do everything these days, but they should aim towards keeping up with all of the team’s responsibilities. Not doing so, will eventually slow the group down and create management issues because person A doesn’t do X. Those cases happen very often because every team has a different setup, and there are always new people onboard, some with less experience, others with more. An SWE with the right attitude, will make the most out of the situation and ask the team for mentoring or training in order to close the gap and acquire new skills.

Automate repetitive tasks

#

Because of the multidisciplinary setup that we’re creating in modern software engineering teams, a significant amount of the things we do today is not building software, and almost on all cases they can be improved by adding some automation magic to it. Every time we’re automating something, be it a deployment procedure, data cleanup, backup, testing or customer invoicing we are creating more time for people to focus on other tasks. It is ultimately an optimisation of the company’s resources and will result in overall better performance. Because we’re already comfortable coding and using APIs, creating small programs to deal with repetitive tasks can be a low-hanging fruit that immediately brings added value to the team and to yourself. Every case is different of course but as an SWE, you should always try to get rid of repetition and routines, as its a waste of your skills and the team’s availability.

Final remarks

#

The principles discussed above resemble several industry trends like the DevOps culture and Site Reliability Engineering, which i purposely avoided in this article. These are the reflection of having worked with multiple individuals and companies trying to adopt it, but presented in an unstructured, not scientific way, which i believe is more tailored to my personal opinion. Over the years I’ve been working on all of them to improve my value as a software engineer and I feel like it paid off, hence I decided to share them hoping that they can help out other fellow engineers. I also expect that as good SWEs, readers go beyond this article and get more input from either other peers or state-of-the-art literature, so that you can make your own opinions based on multiple inputs, maximising your efficiency in becoming a more complete software engineer.