Notes from Google Test Automation Conference 2015

Michal Jablonski

November 24, 2015

The Google Test Automation Conference (GTAC) is an annual conference organised by Google. The great thing about this two-day conference is that it is free of charge, but only selected attendees are invited to join.  This year Google received over 1,350 applications and only chose around 200 participants, including myself. At Vector Software, we are innovative thinkers and are always searching for new test automation solutions; therefore I was very pleased and appreciated being part of this limited audience at Google’s office in Cambridge, Massachusetts.

GTAC is one of very few conferences specifically targeted for engineers interested in testing, compared to, in my opinion, hundreds of general meetings about software development. This year there were great presentations about test automation in many areas, from very low-level testing such as simulating sensors on hardware boards to high-level iPhone apps testing.

The opening keynote presentation was by Jürgen Allgayer from Google, who talked about testing YouTube. This is not trivial when you think that YouTube is not only a website but also a platform for different applications on various devices including phones, tablets, game consoles and TVs, and finally testing the whole infrastructure. One statement from Jürgen that stuck in my mind was his personal favourite goal – “no more manual regression testing”, as there is no time for manual labour, but there are tools to help you with automation, a statement with which I couldn’t agree more.

Hans Kuosmanen and Natalia Leinonen from OptoFidelity put on a great show, presenting a robot operating touch screens such as those on phones, in cars and medical devices. I knew that robots were used in test automation, but I must confess that I never thought they could be used for testing something as precise as touch screens on medical equipment.

After lunch there was a presentation titled “How to Component Test Soup Dumplings” done by Toni Chang, a tester from Google, trying to explain why component testing is important. She mentioned that it was always hard for her to convince people to do this type of testing, so she decided to explain this method using soup dumplings as an example (can anyone imagine doing “system testing” on every restaurant order by eating a sample – one dumpling – before serving the order). If you have the same challenges as Toni in teaching and explaining the importance of component tests, I highly recommend that you watch her segment here.

Brian Gogan tried to list all the challenges that that his team faced during testing Chromecast that were similar to the challenges that the Internet of Things (IoT) industry encounters: How do you test wireless connectivity? How do you monitor hundreds of devices? How do you connect these devices to one system? Those are some questions that Brian tried to answer by showing his team’s approaches. He also advocated parallelisation as a solution for running hundreds of test cases, because, apparently, his team had over a million of them (I wonder if he heard about Changed-Based Testing as a means to speed up testing?). The conclusion from his talk was that the main issues IoT has today are automating IoT testing.

A key presentation was “Your Tests Aren’t Flaky” by Alister Scott from Automattic, whose motto was “insanity is running the same tests over and over again and getting different results”. Alister advocates that the problem is not flaky tests, which return different outputs every time you run them but rather the source code itself. Often development code is hard to test or unstable, and maybe we should spend more time on creating more testable systems. I, as well as most of the attendees, couldn’t agree more with Alister – creating great, bug-free systems is a symbiosis between testers and developers rather than a competition.

Concluding the first day of GTAC 2015, we had some great presentations given by fascinating speakers about important aspects of testing, especially test automation. I definitely learnt a lot and am eager to use this knowledge and experience in helping our customers with better testing and delivering high-quality products.

Michal is a Field Application Engineer for Vector Software. Previously he was a Software Engineer at Intel where he worked on USB 3.0 and mobile broadband modem embedded software. We are pleased he was selected to attend this exclusive event and was able to share his experience with everyone. Be sure to check back soon for Michal’s second blog post from Day 2 of GTAC 2015.

Featured Download
Google’s significant contribution to software development best practices along with unlimited development resources cannot be underestimated. However, many companies don't have the enormous software development resources that Google does. Additionally, there are some limitations to Google’s framework, especially when it comes to meeting the rigorous requirements in aerospace, automotive, industrial controls, medical, transportation and Internet of Things (IoT) markets. Download "Six Ways VectorCAST Enhances the Google Test Model for Embedded Software Development” to learn more about how you can extend and enhance your test framework.