DECLARATIVE UI TESTING @ MICROSOFT

Ed Triou,  Zafar Abbas,  Sravani Kothapalle

Microsoft

edtriou@microsoft.com


Abstract

Software UI (user interface) testing and verification is a challenging and costly problem.

Today, even with the best-of-breed UI automation tools at our disposal, Microsoft (SQL Server included) has heavily invested (resources, time, and money) in testing our increasingly rich (graphical) user interfaces. At the same time many organizations have done ROI (return on investment) analysis and are realizing that; if they had just spent the same amount of time using their product that they spent on engineering automation; they would have had a higher quality product.

There are two fundamental problems that continue to plague today’s automated UI testing. Testing UI continues to be more focused on automation primitives than the richness of the scenario(s). Likewise, now verifying the automated UI has become as costly as the product under test.

Instead we purpose; Declarative Testing – focus what you want to achieve, not how to obtain it.

If you focus on what you want to achieve, you're focusing on the purpose (i.e.: richness) of the scenario, not the hundreds of operations on how to achieve (automate) it. That’s exactly what customers do with a vast majority of our products; they focus on wanting to; design, draw, or build ‘X’. The hundreds of operations (clicks) to make that happen is merely a side-effect (i.e.: implementation detail) of getting what they really want.

By defining scenarios declaratively (in terms of the ‘what’), this also serves as the resultant answer of the product state. Verification could thus be greatly simplified if you already knew what the answer should be.

In this paper, we describe how we leveraged and applied declarative testing concepts within Microsoft; specifically for the Ado.Net Entity Data Model Designer team. http://msdn.microsoft.com/en-us/library/bb738482.aspx