Mobile Ordering solution tested from distance
In the mornings, millions of Americans hurry to work and want to buy a cup of good coffee for pampering. Philz Coffee, a company specializing in custom blends, came up with a mobile app that allows customers to pre-order their coffee. Testing the application became a success story for FOB Solutions.
- Quality Assurance through Functional/Non Functional Testing
- Planning and managing all quality related activities
- Partner beyond Quality
Tools & Technologies that were used
Linux terminal operations
jMeter / jMeter plugins (for results aggregation)
Bug Management – Jira
Test Planning – MindMeister
various customized Excel solutions for test management
iOS/Android Applications in ReactNative
OWASP ASVS level 2 standards were followed
Philz Coffee is an American coffee company and coffeehouse chain based in San Francisco, specialising in customized blends of coffee & tea as well food products. As the number of coffee lovers seeking special tastes grows, the chain with 52 stores wanted to avoid long queues. In 2016 Philz Coffee together with technology partner Work&Co started to work on the concept and planning for an Order Ahead Application for mobile platform. Client was looking for QA support, which would encompass Functional, Integration, Performance and Security testing for the application.
Trustworthy and loyal partner
Sten Vaisma, Team Lead at FOB Solutions tells that FOB was contacted mainly because technical testing was not the area of expertise for their existing partner. Also the project scope & complexity required another layer of QA, acting as an independent & unbiased quality gate before going live. FOB had proven themselves as trustworthy and loyal partner from previous cooperation with Work&Co and already knew the team partially. The strength of the Estonian enterprise was also noted by the fact that they had previously worked in the US market.
Approaching the challenge required very precise planning by the partners. “We had many stakeholders in the project: we, the end customer Philz Coffee and the development companies in New York and Kiev, all working in different time zones.
Our collaboration model became quite close using the whole 24 hours very efficiently” tells Vaisma. In the planning phase overall QA implementation had to fit into existing development pipeline and estimations for monthly scope of work had to be set up. Our work started with QA planning, fitting testing into tight timeline of activities done by already 3 teams. For precise work execution, a detailed Test Strategy & Test Plan was also created.
Test management covered such areas as improving test process and leading test team, while creating test deliverables and supporting documentation. Various QA activities also needed continuous planning and management. “The approach to testing in our case is very agile and experience based. Our strength is that we can adapt as quickly as possible,” notes Vaisma. The consultancy part involved giving feedback about improvement ideas on functionality/design reviews for resulting better product for end-users.
“We conducted both manual and more technical types of testing on this project,” says Vaisma. The Functional, Integration, End-to-End and Design testing from the end-user perspective was guided by Context Driven testing principles and Exploratory testing approach. FOB Solutions QA team of 2.5 FTE also ran non-functional tests for Performance, Load, Concurrency and Security Testing. Vaisma considers the project as exciting, mainly because of the large amplitude of testing:
“One might think, how complicated is there to test ordering a cup of coffee. I can easily say it’s the most complicated project I’ve worked on, from both, functionality / testing perspective. We were testing the complete ordering cycle including Customer Application / Barista Application and several administrative tools.”
Testing in a smart way
FOB Solutions uses an expression called “Smart Testing” whenever they talk about assessing quality through Manual Testing activities. This means that we consider testing as an intellectual process rather than facts checking – quality goes way beyond that. “Testers should be given the freedom to question the product beyond requirements. Daily work should not be driven by following step-by-step Test Cases. Software is way too complicated to afford doing that anymore. Instead of following steps we prefer structured exploration!” tells Vaisma.
Communication with Philz Coffee and all the rest of the team was challenging, but well managed. Mainly because working across different time zones leaves little to no time for discussions or meetings. “Initially, we put together a very precise communication model and used time as efficiently as possible. However, when having the next milestone behind the corner, close-collaboration of development and testing did reach to night hours. That’s not something unusual – the whole team just goes the extra mile!”
Test Methodologies and approach applied by FOB decreased delivery times and increased overall value point for a customer. “This can be noted as one of our strengths compared to our competitors on the QA field,” tells Vaisma. Ultimately this means that testing itself was almost never a bottleneck for major milestones leading to faster business conclusions and time to market. This also resulted in increased product quality post launch and minimal hotfixes during maintenance phase. “The clients revenue increase resulted mainly due to launching the product itself and therefore increasing customer base, monthly active users and amount of orders,” says Vaisma.