Interview-with-Ben-SimoBen Simo
Organization: eBay Enterprise Marketing Solutions
Role/Designation: Software Engineer, Tester & anything else the business needs
Location: Arizona

Ben Simo, also known as QualityFrog, is a context-driven tester and agile software developer who has been practicing his craft for over two decades. Ben views software testing as a search for knowledge – both confirming existing beliefs and seeking out that which is unknown. As a tester, Ben strives to enable teams to make informed decisions. Ben is driven by a love for people – people who are impacted by software. Ben seeks to help create software that enhances rather than complicates people’s lives. Ben seeks to first understand the specifics of each situation and then select practices that fit the context. Ben has applied his testing skills for companies large and small; spanning a variety of industries, including: defense, healthcare, finance, education, Internet services, and advertising. Ben operates IsThereAProblemHere.com – a blog that provides testers with an opportunity to laugh and learn from wild-caught software failures.

* Interviewed by Jay Philips

1. Tell us about your journey to becoming a software tester. How did it start and how this has been so far? Was it planned or by accident?
Twenty-two years ago, I fell into a testing role with the illustrious title of “data collector”. I was one of a few hired to execute test cases created by others and report the results. However, from day one, testing didn’t go as planned. Executing the test cases required deeper understanding than was communicated in the documentation. I quickly learned that testing can be fun and mentally challenging work. I discovered something that I enjoyed more than programming: figuring out how and why existing things work; and discovering ways in which they don’t work. Testing also provides me an opportunity to continue to write code and do other technical work that I enjoy.

2. When did you realize your passion was software testing?
I learned that I loved testing during that first testing project. However, I didn’t yet realize that testing was a skill set, and could be a career, of its own. I saw myself as more of an expert in the subject matter of the bulk of my testing – digital imagery transmission – than a tester. When I moved from government to commercial work, and was first given a title that included the words “quality assurance”, I entered a phase during which I became a bit of a process and quality cop. I learned the hard way that quality is about much more than compliance with process and specification documents – that quality is in the relationships between software and people. In 2004, I discovered Bret Pettichord’s original Four Schools of Software Testing presentation. This introduced me to the principles of the Context-Driven School which helped me better understand my evolving thinking on testing. Soon after, I became involved in the Context-Driven community and had my passion for testing re-ignited.

3. Do you regret being associated with software testing today? Given a chance would you move from testing to any other field in IT?
That’s an odd question. Why would I regret being associated with software testing? Good software testing is needed and appreciated. However, many have been exposed to things called “QA” and “testing” that don’t provide much value. This has given some the impression that testing isn’t a valuable specialty of its own. Each of us can help change this perception by providing testing services that our stakeholders value. Get to know what matters to your stakeholders and apply your testing knowledge and skills to help them be successful.
I have the opportunity to move away from testing at any time; and I spend a great deal of my work time doing things that many may not consider to be testing. I design and build software tools. I fix bugs in the systems I test. I investigate problems. I analyze data. I support customers. I help solve problems. I apply my testing skills to all of these things. No matter my title or primary role, I don’t envision myself not testing.

4. You describe yourself as an amphibious time-travelling context-driven cyborg software tester. What does this mean?
I am amphibious. Although I am a software tester first, I’ve spent much of my career straddling roles and environments. It is common for testers to view me as a very technical person while programmers view me as one more focused on value. I’ve often found myself playing the role of liaison between technical and regular people — helping disparate stakeholders understand one another.
I am a time-traveller. However, my time machine only goes forward. In my twenty-some years of professional experience, I’ve seen many technologies and practices come and go, and then come back again. One thing remains constant: the need to test whatever it is that us fallible humans try to create in the virtual worlds inside our machines — virtual worlds that have real impact on real people in the real world.
I am context driven. This means I reject the concept of best practices. Instead, I try to first understand, and then select practices that fit each situation. I adapt my testing techniques and practices to the context. While my context-driven worldview is often at odds with those of others with whom I work, I recognize that people are the most important part of any context, and therefore try to find ways that I can contribute within whatever environment I find myself.
I am a cyborg. I have both human and artificial parts. I enhance my sapient brain-engaged testing with automation. I use automation to enhance and extend my capabilities as a tester.
I am a tester. Most of all, I am a tester. While I have many other skills, I generally use them to support my testing and help others test well. I like to play the part of investigator and bring people information that enables them to make better decisions.

5. You’ve been testing the Healthcare.gov site and found some very critical security issues. What do you think could have been done to avoid the issues you found?
The security issues I’ve identified are the sorts of problems that those who design and test for security would have most likely prevented, or detected and fixed, prior to release.
For example, the system contains many Internet-facing REST methods for retrieving information about users without any authentication. This enabled automated collection of real people’s email addresses, usernames, password reset codes, and security questions. No single vulnerability exposed all this information, but chaining together a group of vulnerabilities permitted access to information that should have been protected.
The security issues are the types of things that are usually prevented by:
• making security a priority in software design and implementation and testing
• ensuring developers understand how to build secure systems
• keeping communication open and ongoing between those building and integrating the various system components
• engaging in threat modeling: considering angles of attack and building appropriate defenses
• engaging in systems thinking: considering how the parts of the system (including people) influence other parts and the whole
• testing the parts and the whole with a focus on security
In the case of Healthcare.gov, I believe the security issues are symptoms of bigger problems — primarily management problems. However, I also wonder if anyone directly involved in building and testing this thing spoke up loudly and tried to warn others of the potential harm in going live with insufficient testing and known security concerns.
Some of the issues I’ve encountered with Healthcare.gov have been fixed; others haven’t. I am not yet convinced those who built the system understand how to create secure software systems. You can read my bug reports at http://blog.isthereaproblemhere.com/search/label/Healthcare.gov

6. You run the website “Is there a problem here”. What made you start that site?
I started IsThereAProblemHere.com to provide software developers and testers real world examples of misbehaving software. I started the site to help encourage people to look beyond their written specifications and happy path testing. I want to provide illustrations that problems aren’t just technical — they are things that impact how people relate with the software systems we build and test. I want to provide an opportunity for people to apply their testing heuristics to software that others considered good enough to release. I believe we can all learn from failure — and I prefer to first learn from the failure of others.
Oh, and sometimes failure is funny — when it happens to someone else. Many of the problems on IsThereAProblemHere.com are more entertaining than educational. 🙂

7. According to you, what is lacking in today’s commercialized software testing industry, especially in test management?
I have never been a fan of most commercial test management tools. I find that many of them overly complicate testing – they provide a forum for much busy work that has little to do with testing.
My biggest concern with the commercialized software testing industry is the focus on process and tools. While process and tools are essential in the work we do as testers, they play a support role. When we let process and tools drive, it becomes easy to forget that the most important element is the tester’s mind. Tools extend and amplify our existing knowledge and skill – so let’s first seek to become competent testers.

8. What qualities will you look for in a candidate when you want to recruit someone for software testing job?
I first look for insatiable curiosity. I look for people who aren’t satisfied with what is already known. I look for people that seek to understand what matters most, and to whom it matters I look for independent thinkers who are willing to challenge existing beliefs, but also strive to accurately report their findings in a way that stands up to scrutiny. I look for people who are technically competent for the job. I often place more importance on an ability to learn and apply new technologies and skills than on the specific technologies and skills the job requires. Technologies change rapidly in this business; I want someone who can adapt. I prefer people who are thinkers over those who memorize and regurgitate information with little depth of understanding.
Finding these qualities can be difficult. There is no simple exam or set of interview questions that can tell me if someone is a good tester. Aptitude and skill for being a good tester is best demonstrated — something that is often difficult to do in the typical job interview process.

9. What will you suggest to people who want to join IT industry as software testers?
First, become a self-directed learner. Testing is investigation and learning, and communicating what has been learned.
Learn about the nature of software. Learn about how people relate to software systems. Learn to test. The Association for Software Testing’s BBST series can help provide a good foundation. Learn to recognize problems. Get lots of experience using and observing both good and bad software.
Learn to program. Become technically competent, but don’t limit yourself to technical study. Learn to adjust your focus from a narrow technical view to a bigger systems view.
Learn about things other than technology. A tester with breadth of knowledge and experience is often more valuable than one with very deep knowledge in few things. Too narrow of a focus in our learning makes us dull. Variety in learning stimulates ideas.
Learn to examine things critically; seeking evidence and understanding as you go.
Learn to communicate — both orally and in writing. A tester who is skilled at discovery is of little value if they aren’t equipped to convince others to act on their findings.

10. Name few people you would like to thank, people who helped you directly or indirectly in your career as a software testing professional.
As soon as I start naming people, I’m going to end up leaving someone out to whom I owe as much or more than those I don’t name. I am thankful for all the people with whom I have worked, argued, commiserated, or interacted with in any other way. The people with whom I have interacted have helped make me what I am today. Some of my worst professional relationships have been some of my best learning experiences. I’ve also had some great mentors along the way. I appreciate all of you – even those I don’t like.
Oh, this reminds me of one more thing that those interested in becoming great software testers should do: Get to know the people in your neighborhood. Software development and testing is as much a social activity as it is a technical activity. Get to know your stakeholders. Get to know the business people. Get to know the developers. Get to know your customers. Get to know those who are impacted by the software you test. Also, get to know other testers – both within and outside your current place of employment. Engaging in a professional community is one of the most important things you can do for your testing career.

Blog/Site–
IsThereAProblemHere.com
QuestioningSoftware.com
Twitter ID – @QualityFrog

 

https://i0.wp.com/www.testingcircus.com/wp-content/uploads/Interview-with-Ben-Simo.jpg?fit=450%2C432&ssl=1https://i0.wp.com/www.testingcircus.com/wp-content/uploads/Interview-with-Ben-Simo.jpg?resize=150%2C150&ssl=1Testing CircusInterview with TestersInterview with TestersBen Simo Organization: eBay Enterprise Marketing Solutions Role/Designation: Software Engineer, Tester & anything else the business needs Location: Arizona Ben Simo, also known as QualityFrog, is a context-driven tester and agile software developer who has been practicing his craft for over two decades. Ben views software testing as a search for knowledge –...