Do you think that the words program and software are interchangeable? If you do, then you probably aren’t a software engineer.
Though these two terms may be synonymous for the average consumer, a program is to software as a drive shaft is to a working automobile. Continuing with this metaphor, the programmer is to the software engineer what the assembly line worker is to the car’s designer.
This isn’t a perfect metaphor (software engineers will often work on code and programmers may work on design components) but it’s the basic idea. Programmers write code. Software engineers work on creating a desired functionality in a piece of software (photo editing, data storage, instant messaging, etc.) through the use of programs and other pieces of software/system software. This will often involve coding, but an engineer will be able to both design and write this code.
The software development process is divided into distinct stages, mainly: requirements, design, implementation, verification and maintenance. Some engineers think that “Waterfall” methodology is on the way out, and many companies are using an “Agile process,” wherein small portions of the software are independently designed, created and tested before moving on to complete another part of the software.
However the workflow of your company is broken down, the stage or stages that your software engineer will be working on will determine the skills that you need to look for in the interview as well as the tests that you should give to your engineer candidates.
Getting What You Need from the Interview
To test how effective a candidate will be on the job, the first step is identifying what the person in your software engineer position will be responsible for. Generally, there are 4 levels of skill for programming and software engineering positions.
E1: Designs on a small scale and codes designs provided by senior engineers.
E2: Can design small softwared features, coordinates work of/ train E1’s and will work more closely with E3 and E4 engineers.
E3: Can operate independently and can be the lead engineer on a project or a sizable piece of the software. Works with E4
E4: In charge of multiple teams, the completion of the final product and aligning this product with future plans.
If you aren’t sure where your software engineer job falls on this spectrum, ask a software engineer! While you don’t need to classify your open job in these terms, being specific will help you to attract the right sort of candidate. For instance, while some talented E1 engineers might have the talents to perform in an E2 position, you wouldn’t want them applying for your E3 or E4 positions.
Once you know the seniority level of the engineer you need, the next step is identifying which parts of the development cycle they’ll have a hand in:
Software Requirements: Identifying the requirements of stakeholders as well as requirements for software functionality.
Software Design: Designing the systems and programs that result in these requirements being met by the software.
Software Verification: Investigating the quality of this software and whether it meets stakeholder/functionality requirements.
Software Maintenance: Fixing issues identified in the verification stage.
Once you have these candidate requirements squared away, you can start creating a test that will reveal these requirements in your applicants.
Interviewing and Testing Software Engineers
Though all of the candidates who reach the interview stage should technically meet the minimum skill requirements of the job, we’re sure you’ve all felt the sting of this assumption.
In order to save everyone time, the first skill test that you give to software engineer candidates should be over the phone. This first test should test for the basics, like whether or not they can use the coding language you need them to. After you’ve weeded out the people who have no idea what they’re talking about (there may be more than you’d think) you can continue with the interview process.
Some companies will have tech candidates complete tests on a computer or even a white board, but this is not always necessary. Doing a problem in front of someone or on a time limit adds stress to the application experience so, unless pressure resistance is important for the job, you may want to let applicants complete tests on their own time.
Here are the different areas of the software development cycle and the tests that should be applied to engineers who will be working on those stages.
Though the job of gathering and communicating stakeholder requirements is often the job of a product manager, an engineer will ultimately be responsible for translating these requirements into software that meets these requirements.
To test engineers in this area, ask them to tell you how they’d “create a piece of software that does ______”
A good software engineer will ask something like, “OK, but what do you really want?”
Good software engineers will recognize that they don’t have enough information from your initial question and will want to know the scope of the requirements and the context in which the program will be used. The way that a candidate interprets this question will demonstrate their abilities to think in terms of software requirements, both on the functional side and in the context of the user. A good engineer will be able to take these requirements and lay out how a piece of software would result in these requirements being met.
How do you organize requirements?
Do you prefer forward requirement tracing or backward requirement tracing?
How do you deal with requirements that are likely to change?
How do you deal with requirements that are unclear to you?
Though the design is always dependent on the requirements on a piece of software, knowing what a piece of software needs to do is still a long way from designing that software.
This area of software development is divided into front end and back end design. Front end design is designing the features that will be experienced by the user while back end design is designing the technical components that will be making the software work. You want design engineers that are suited to their side of the equation, but you also want engineers who are looking at the whole picture and who are adamant about everyone being on the same page. Communicating at this stage is crucial, so you will also want to test design engineers for the way that they work with their peers on projects.
Along with these high level front end questions, you should also test engineer candidates for specific competencies (Java, C++, cloud technology, data storage/structures, networks, etc.) that will be necessary for designing your company’s software.
How would you improve the user’s experience during a long loading time?
How would you improve on the experience of using our company’s latest software?
What are some techniques that you’ve used for prototyping an application?
How do you ensure that front end and back end structures are aligning during the design process?
Back end is concerned with the nuts and bolts of your software. Along with these general design questions, have some of your tech employees draw up questions to test for specific competencies (Java, C++, cloud technology, data storage/structures, networks, etc.) and the other hard skills they will be using on the job.
What kind of diagrams have you used when designing software?
How would you adapt your design for a mobile based application?
What tools do you use when evaluating the performance of your code?
How do you ensure that front end and back end structures are aligning during the design process?
Implementing the plans from the design stage is where things start to get hairy. No matter how much you plan, there will always be problems, and a good construction engineer will be able to work through these problems as well as having solid understanding of coding. Apart from testing coding ability, you should also ask questions that will reveal crucial software construction experience:
Can you walk me through your process of completing a piece of code, from requirements to creation to delivery?
Can you tell me about a time that you predicted a change that needed to be made during the constructions stage?
What is your process for reviewing a team member’s code?
How do you ensure that your code can handle different error situations?
What programming language are you most comfortable with? What is the major limitation of this language?
Software verification is the stage at which the tough questions are asked. Does this piece of software meet the high level requirements put forth by stakeholders? Is the software we built the best way to meet these requirements? Are there any bugs or gaps in functionality?
You won’t always be looking at the answer you want in this situation, so it’s important to have engineers who are concerned with product quality, not getting a project over with or just looking good. You want engineers who are focused on delivering a product that meets your requirements and who aren’t afraid to say when the final product falls short.
What types of problems have you encountered most often in your products after deployment?
What is the difference between functional testing and exploratory testing?
Can you tell me something you learned about testing and quality assurance in the past year?
What was the largest problem that you’ve discovered in a piece of software? How did you correct this problem?
What is your process for de-bugging? How do you prioritize?
Maintenance is a crucial step in the software development cycle. There will always be bugs, but discovering and fixing these bugs in a timely manner can make a huge difference for the popularity of your product. Along with de-bugging, maintenance involves working on the updates that are crucial for your software to remain on the cutting edge. You want maintenance engineers who have a good deal of experience working with and updating software that is similar to yours.
To you, what are the most important tools to use when monitoring software during maintenance?
What are the most important things to keep in mind when performing maintenance on a product that is in use?
What steps have you taken to make software more easily maintained?
Why is maintenance commonly one of the most expensive phases of a product’s lifecycle?
VP of Marketing at Digital Astronauts
Ronny Cheng is one of the Co-Founder's of Digital Astronauts and has helped drive lead generation in the software industry for organizations of all sizes -- from start-ups to Fortune 500's. He helped build one of the first online software review websites, specializing in CRM, ERP, and HR software. He's a nationally published author with extensive experience working with the HR/Recruiting industries largest brands. In his spare time, you can catch him on Instagram doing his best food blogger impersonation.