elmarschraml.com

Occasionally updated personal site and blog

Monthly Archives: February 2022

Some lessons from 10 years in the business of IT consulting

These points are unsorted, random and subjective. Quite a few are negative – not because I want to focus on the negative, but because we learn from our mistakes (or the mistakes of others). 

If your sales are slow, it might be that you are bad at sales, or there is a downturn in the industry. Though most likely nobody needs what you are offering.

The more pushback you get on price, the more it is a sign that you are in the wrong business. A great business is one where conversations go like this:

Client: “How much will it cost?”

Us: “Here’s a number that’s pretty high, because you’ll probably want to negotiate a rebate—”

Client: “Cool, when do we start?”

If they ask you to fill out hundreds of questions, submit a detailed calculation, and submit solutions before starting, they either don’t know what they are doing, or do not really value it. Either way, you don’t want their business, and you wont get it. 

Firing someone can be great for morale. If it is someone where everybody they have worked with is annoyed with their incompetence. 

Any truly major decision or change takes about 3 years to have its full effect. Your most important partner got bought, and is now a completely different company? Projects are managed badly, and the employees need to compensate with unpaid overtime? Management targets are set unreachably high, and bonuses never paid out? A new line of business, with a brand-new team? A new CEO? Anything that happens in the first year after that change is not yet a good indicator of how that will work out. 

Profits are a lagging indicator. Nothing ruins a company as surely as incentivising management only on this year’s profit. It is easy to increase profits, all you have to do is cut investments. 

You need a healthy ratio between junior and senior people. Too many seniors, and junior people never get to stretch their wings. Not enough seniors means there is nobody to teach the juniors. 

If you tightly control your people so they only spend time on things that have a measurable impact on the bottom line, you avoid a lot of waste and slacking off. You also get rid of serendipity, research, improvements to internal tooling, cross-pollination between departments, experiments and all the other things smart, driven people will do when nobody is watching.

If you invest in a new line of business when it’s uncertain, it might not pan out, and be a waste of money. If you invest in a new line of business only when it has become a sure thing, it will already be crowded with competitors who invested before you. Also: an uncertain partner who needs you will be willing to support you, and remember you for taking a chance on them. An established partner has their pick of companies, and you have to be the one to beg them to take you on. Also: when a market is new, nobody has experience or references, so that’s OK. When a market is established, you face the chicken-and-egg problem of gaining experience when you have none. 

For the really important tasks, you want to send your best and most experienced people. But if you only ever hand big important tasks to the most experienced people, the junior ones will have no chance to become experienced. Sometimes you have to ask people to step into footsteps that are a size too large for them, for them to grow into those. But only one size, not two. And you have to accept that sometimes they will fail. But if you don’t, you will have no experienced people, and be forced to have people step into tasks that are two sizes too large for them. 

When there is a conflict between what an employee wants, and what the company needs, do a cost-benefit analysis. For example, paying for a  training that is not directly relevant to the company may cost 2000 bucks, but that’s a cheap way to buy motivation. 

You know how external rewards can kill intrinsic motivation? That goes the other way round, too. The more you force people to go the extra mile (without extra reward), the less willing they will be to go the extra mile when nobody is watching. 

If you have to fire people during their trial period frequently, you either suck at recruiting, or you suck at onboarding. If you never have to fire people during their trial period, you are too conservative and too picky when hiring – give some uncertain, wildcard candidates a chance. 

There is no such thing as a perfect candidate – some lack experience, some come from a different domain or tech stack, some seem a little weird, some are demanding,… Figure out what flaws you can accept. And judge on whether you see potential for improvement. 

The most loyal employees are the ones to whom you showed loyalty first – the highschool dropout who you took on as first an intern, and then as an apprentice. The trainee who had almost no programming experience, but a burning desire to learn. The star performer who burnt out, and needed months to get back to being productive. Those will stick with you through tough times. 

It is very easy to increase profitability quickly – all you have to do is reduce investments, cut training, and mandate unpaid overtime. A bad manager will commend you for it. A good manager will realize that in the long run this is disastrous. 

When a project is well and truly fucked, and a hopeless disaster, the most important thing to get back on track is to give the impression that you know what the problems are, you have steps to tackle them, and you have a timeline and priorities. It can be an incomplete plan with imperfect solutions and an uncomfortably long timeframe – but knowing how bad it is is much, much better than fearing the worst. 

Fast, cheap, good. Time/Money, Quality or Scope. One of them has to give if the other two don’t work out. If a client has unrealistic expectations and is at the same time insisting on being inflexible on all three dimensions, no amount of project management can make him happy. 

It is much easier to edit a mediocre solution, than to sit in front of a blank slate. The same goes for a client. Never ask “what do you want us to do here?”. Always ask “what result do you want to achieve with this”, and then “I would suggest the following – does that work for you?”

You are judged on results, not effort. Unfortunately that also goes for projects: If you take on an easy project that basically runs by itself, and even a clueless intern could not mess up, and barely make it work, you will be praised for your success. If you take on an impossible doomed project, and by heroic effort almost pull it off, you will be chastised for your failure. A good manager will reward heroes. 

How many people leave is less important than why they leave. If they want to move to another city, or change industry, or tech, or role, fine. If they leave, and in their next job they do the same work with the same tools and for the same clients, just at a different company – they were not attracted somewhere else, you drove them away. If they pull other people with them, it means they really like it a lot better elsewhere. 

Organizational design matters. Somebody whose work is measured in the same P&L sheet will be more helpful to you than the same person if they are not. 

You do not need the highest pay, the fanciest office or the most comfortable job, if you can offer interesting work, great teams, growth and appreciation. How do you convince a candidate to come work for you, after he has told you that he has another offer that pays 20% more? THAT is your employer brand. 

Never save on equipment. Never save on training. 

Nothing makes your boss happier than putting in extra effort without him having to ask for it. Nothing makes employees happier than treating them well without them having to ask for it. 

Do not try to make your team happy. Tell them what you need them to do. Tell them why it matters. Give them what they need to do it. Reward Successes. Happiness follows. 

If you say no to additional work because your plate is full, you make people unhappy now, but they will respect you, and you get to make the decision of what to drop. If you say yes to additional work when your plate is full, you will make people unhappy later when you either drop the ball or half-ass it, they will resent you for being unreliable, and your stress level makes the decision of what to drop for you. 

Experiments that fail are OK. Experiments that fizzle out are a failure. Have a good balance between cash cows and experiments. 

Hiring developers? How to do code reviews in technical interviews

A significant part of our technical interviews is having the candidate do a code review on a piece of example code. I’ve written before about why we do that. Today I’ll share a few tips on how to make it work well.

The code

The code you give the candidate to review needs to be long enough to include enough things to talk about, but short enough to understand quickly, and keep in your head. The code we are currently using is a little under two pages when printed out. We could probably strip some of the boilerplate and go to one page – but anything less than a page is probably not enough.

It should NOT be a complicated problem domain – the review will be about the code, not the specific problem the code solves. But ideally it belongs to an industry your company typically works in. If you get a candidate that has worked in your industry before, you also get to see how familiar they are with topics and problems typical for that industry. If they have not, I of course expect no knowledge about that domain – but a senior candidate should realize where they need to ask for domain knowledge.

Do not use code that implements a complicated algorithm – you don’t want candidates to spend their time trying to understand it. But for most candidates it is easier if what the code does actually makes sense. At first I used something where the business logic was intentionally nonsense in order to make clear that it was not about the logic, but a lot of candidates actually got stuck trying to make sense of it. So now I’ve streamlined the logic to the point where it is still dependent on unknown requirements, but seems consistent.

It works best to pick a class that you wrote for a real project, strip it down to a suitable length, and then intentionally mangle it to include lots of questionable code that should get flagged in the review.

The issues

The most important thing is not to prepare a quiz (“Find the 10 mistakes hidden in the code”), but use the code as an invitation to start a discussion.

Include lots of different things that might draw the candidate’s attention – from commenting to formatting to syntax to logging to exception handling to null safety to naming to nested method calls to method length to duplicate code to performance. This makes it easier for candidates to find something to start the discussion with. And you also get to see not only what they notice, but what KIND of things they notice – a candidate that talks about ways to optimize performance by inlining a method call or using a different list implementation is probably a completely different kind of programmer than someone who focuses on method length and variable names.

Include issues that are suitable for different levels of experience. For people straight out of university, I include a few language constructs that are fairly basic and taught in any university course, but rarely used in real life. For experts in the particular language, I include some obscure syntax constructs that most people won’t recognize – but I’m delighted whenever somebody does. And for experienced programmers coming from other languages, there are a few deeper issues that are language-agnostic but should be red flags for candidates that are experienced enough that they have probably been bitten by that before.

Most of the review works regardless of what language a candidate is used to. It makes no sense for a Lisp programmer to review a piece of code written in Java – but for somebody coming from a background similar enough that the code is recognizable, we just ignore anything language-specific and focus on design and general coding principles.

The most fun topics are those where the code is not necessarily right or wrong, but subject to either personal taste or project requirements. With a senior candidate, I frequently even get new ideas or learn something new. 

If you are working in a specific industry, you might also include some issues where somebody with experience in that industry could be expected to recognize some typical problems or questions. For example, when your review code deals with testing, it could have some areas that are likely to cause false positives. Or if you are dealing with payments, an experienced candidate should probably bring up issues like fraud or interrupted requests.

Doing the review on paper, with everybody having their own copy, actually works great, and is much easier than dealing with projectors or screen sharing. Just be sure to print in color, with good syntax highlighting, and to include line numbers so everybody can follow along. If your co-interviewers are not yet familiar with this interview style, or the particular piece of code being used, it’s also helpful to maintain a separate interviewers’ version of the code that includes comments about the things that are intended to be found in the interview. 

Of course, before letting the code to be reviewed loose on candidates, try it out a few times with co-workers at different levels of seniority, both to get a feel for the amount and level of issues you can expect candidates to find, and also to make sure that it is possible to quickly grasp the code and find some issues with it. 

How to perform the code review

Before the review, I ask candidates which language they are strongest in. If the code to be reviewed is in Java, but their best language is python, I’ll skip the parts about syntax or language-specific details. If they claim to be an expert in Java with 10 years of experience, we might end of going deep into the details of e.g. various implementations of List. 

It is important to point out the basic premises of the review, so they do not get lost:

This is not a checklist test, but a pretty open-ended basis for a conversation about code quality.

Feel free to cover any aspect of “Code Quality” you can think of  (for junior candidates, I also mention a few examples like readability, structure, error-safetz etc to help them get started – whereas senior candidates should be able to define code quailty themselves)

It is not about syntax, the logic or algorithm, but about quality

The code is definitely sub-optimal, and you are expected to be critical (especially important to mention when interviewing junior developers, who might be shy about criticizing other peoples’ code)

The best way to introduce the basic premise is a story like this:

“Imagine that you have been working with us for a while, and are now an experienced engineer. This piece of code was written by an intern – he was very motivated, but is also inexperienced, so there are probably many things he missed. You are all that stands between this code and production. 

Your job is to do a quality check, for any aspect of quality you can think of. There is no checklist of “10 things to find” – there are many different things one could talk about, and often also aspects that are open to discussion, or a matter of taste, or depend on context and requirements. This is not about syntax – you can assume the code compiles without errors. You do not know the requirements – this is not about the business logic, you can assume that I ran the code in the most obvious good case, and it worked. 

Feel free to think out loud, and start by just mentioning things that seem less than ideal, or strange, to you, and we’ll use that to start the conversation”

The last point is important – many, especially junior candidates, get hung up on trying to read and understand the whole code before saying anything. It helps then to subtly prompt them with “Which line are you at right now, and what do you think about it?” or even point to something specific like “Anything strange about that switch statement on line 17?”. After a minute or two it then becomes clear what is expected, and most candidates will walk you through the code by themselves. And the most interesting topics often start with an off-hand comment like “hmm, that seems weird…”

Try to turn the review into a conversation, by not just noting issues they found, but talking about how they would improve it, what alternative solutions exist, or even talking about which requirements would lead one to choose that kind of solution. 

15 minutes, including the introduction, is usually enough to get a good signal on the candidate’s aptitude. If you have to stop because you’re running out of time, that’s usually a good sign, that you’re both having fun and interesting discussions. 

Things to check for in the code review

What kind of things do they notice? Are most of their comments along the lines of “this method is way too long, and javadoc is missing”, or more like  “we could optimize performance by saving some method calls here”?

What size of things do they notice? Small details like a missing import statements, or major bugs in the control flow?

How proactive are they? Do they just comment “this code is really bad”, or provide a well-reasoned argument why,  and suggest improvements?

Do they find solutions? I do not want to ask trivia questions about e.g. the syntax of regular expressions, but if a complicated piece of code could be replaced with a trivial reges, it’s good to see that someone has regexes in their mental toolbox, and suggests to use them. 

How deep is their knowledge? You can alway dig deeper on any concept they find. For example, a piece of code where you should really, really have used a Hashmap could be just noted, or the better the candidate is the more likely you can use it as a jumping-off point to go deeper into e.g. data structures, or performance characteristics of sorting algorithms, or the benefits of multi-threading, or the possibility of a bloom filter as a heuristic alternative, or whatever else the candidates has deep knowledge in. 

Do they have good taste? Do they care to make code that is clean, well-structured and easy to understand, or “if it runs, it’s good”?

Are they current? While I do not want to make syntax trivia too much of a focus, it’s always good to include either a bit of syntax that is obviously outdated to see if they suggest replacing it with something newer, or include something brand new to see if they recognize it. 

The range of results between candidates is huge – I have had everything from “can barely even read the code, does not know anything to say about it” to “races through everything I ever thought someone could mention, including the most obscure tidbits I don’t really expect anybody to find, in 10 minutes”. And the best candidates have pointed out flaws I did not even include intentionally, or suggested possible improvements I never thought about before. But even weak candidates can, with some prompting, find at least a few things to talk about, so nobody has to walk out of the interview feeling like a total failure. 

Ideally, the code review is both fun and a learning opportunity for both sides. For me, a job interview should feel as little as possible like an exam, and instead be a conversation among experts to judge each other’s level. Pretty much what I imagine university graduation exams would be like in a perfect world – not “answer these 100 questions”, but “let’s have a conversation, and see if you can hold your own among professionals.” Doing a code review is the best way I have found to get there. 

And if you are doing code reviews in job interviews, I’d love to hear your tips at askanengineeringmanager@gmail.com

Hiring developers? Why you should do code reviews in job interviews

I’ve been involved in recruiting software engineers for a long time, to the point of job interviews taking up a significant part of my time. So I’ve done enough interviews to notice patterns about what works and what does not. The single best thing I changed about how we interview is to iinclude a code review as a standard part of the technical interview.

Why deal with code in an interview?

The most important factor for a developer is, unsurprisingly, that they are able to effectively work with code. What’s less obvious is that there are a lot of people who nominally have the right sort of experience and background by e.g. having completed a degree in computer science, but are hopelessly lost doing actual software development. Others have a different tech background than what you are using,so it might go either way – they might have to relearn their whole way of developing, or just transfer their skills to a new syntax. And the single most important thing in an interview is to not just talk about projects and tech, but get down to actually using it.

What’s the alternatives?

Asking questions about technology often comes down to a quiz about random trivia. But whether a candidate knows a particular library or function by heart is irrelevant, since in a real work situation you could always just look it up. Besides, it’s too easy to mistake “candidate happens to have worked with the same tools as I have, so knows the same details I do” with real, generalized skills.

A lot of companies do whiteboard coding, i.e giving candidates a programming challenge in the interview, and having them write the code to solve it on a whiteboard during the interview. Which is a nice test of stress resistance, but nothing else. Nobody does any real programming without an IDE (or at least a decent text editor) and access to language or library docs. You can improve by providing a laptop having all that preinstalled – but the most fundamental problem with coding tests is that the 30 minutes or so allotted to them is way too little to understand a non-trivial problem, find a solution to it, code it, test it, and bring it into production-ready shape. At best, you are testing how quickly they can whip up a first prototype. At worst, you are only testing if they can come up with the right algorithm under time pressure.

So you might instead give candidates a more significant programming exercise as a take-home exam, to complete on their own time. Which is a reasonable test for programming aptitude – except you don’t know whose, since they might have had someone else solve it for them. Or brute-forced something that’s really too hard for them by spending a whole weekend on something that you’d expect a competent candidate to finish in two or three hours. And those competent candidates will either not bother to do several hours of free work for you, or by the time they get around to it have several competing offers on the table.

A variant on this that seems to work is to take a small but real development task from a real project, have prospective candidates do it in their own time, and pay them for the time spent. This way you get a real work sample, the candidate does not have to work without pay, and both sides experience what it would be like to work with each other. I’d be a little concerned about not being able to compare candidates, since each candidate gets a different task. But if you are like any other software company, you’d probably like to hire as many good developers as you can get your hands on, not just fill a single position. So “How easy is it to work with him? Can he do the job?” are much more interesting questions to answer than “Is candidate x better than candidate y?”. The problem, and the reason we do not do this, is that you need to come up with enough development tasks that are suitable. They need to be small, not time-critical, relatively isolated from other tasks, and require neither significant setup time nor product- or project-specific knowledge – to find all those criteria in a single task basically never happens if you’re doing custom development.

Code reviews to the rescue

So for us doing a code review, by presenting candidates with a medium-sized piece of pre-written code, and asking them to review it, has turned out to be the best way to evaluate technical skills. It does not rely on google-able memorization, and is flexible in how much time you spend on it. And since it is a task that most developers have probably done in their job before, it actually measures real-world job performance rather than training for something interview-specific.

It took a few tries to get it right – so next time I’ll share some of what we’ve learnt about how to do a good code review in a job interview.