i’ve been working on dwh (data warehouse) testing for quite some time now. what started as a fill-up time for a person on leave became a passion and i continue to learn with each passing day.

a green-field dwh project i worked on in my early days was a poor, poor start for me and a failure; from my perspective at least. i had no access to application, data source, business rules, requirements anything. i hated working on that project, but saw that as an opportunity to learn more about my newly acquired passion. i continued to validate what went in to what came out based on my understanding (no, read that what common sense told me) asking people their views, validating my assumptions etc.

as time progressed, the project stabilised and all was well. we moved onto other things that mattered and this one continued to chug along in the background with no changes, defects reported in production.

a few months ago, a decision was taken to replace the old application (on which the said dwh was built) with a new one and we were tasked with delivering the new(er) version of the dwh in addition to the existing one. with me the only person experienced from the previous trauma, i hoped to make a contribution in making the overall product better. i still do as it is far from complete.

the first task was to get the requirements right. being a like-for-like solution, it was crucial to ensure that what was in prod was infact right in the first place. by this time, i was fortunate enough to have a voice and say in demanding (or should it be requesting?) appropriate tools(?) to allow me test the way i wanted it.

whilst the requirements were being churned out, i took time to get change requests approved for access to the original and new application, to their respective databases and started tinkering the systems. create a transaction, watch the data across every single table, generate (or validate, more like) the er-diagram provided and even question the business rules and requirements.

by the time requirements were ready, i got a fair idea of how the application worked and how the data flowed. it was easy for me, now, to dictate how the data got created, how the etl got tested to ensure the cubes, reports would be accurate enough.

most things were new; from the project managers to business analysts to dev team members (which was based offshore); the only resources that were constant were me the tester and a consultant who captured the requirements for the initial dwh solution (and who incidentally was the developer for it too!)

the first release of the etl tanked abysmally. root cause: incorrect interpretation of the requirements; faulty tech spec. like all my defects, i start reporting the cause and suggesting (pseudo)code to fix issues which is welcomed by a few team members (of dev team). sadly, many objected saying test was driving development! and there i was thinking we were now moving in the right direction! not all managers liked this. big meeting was called and a decision made that thats no good. test shouldnt be dictating how this needs to be done. frankly speaking, if i were to be given a chance, i could do the whole thing myself; well, not really, but you get the idea.

those in my support vouched for the knowledge i had and supported my idea of why me being the most knowledgeable (without boasting) they should use the knowledge for the general good of the product and deliver quality code.

the result is that a business analyst (instead of a dev tech lead or tda) is tasked with creating a technical specification. with my help. now, not all ba’s are technical. i agree that to an extent they should be limited to capturing business requirements rather than get into the technicalities of how it is to be achieved. but heigh-ho…

like most ba/s, most of our ba/s are not technical; fortunately in this instant i’m at ease with design, ba work and database so it ends up being on my plate. we (read i) continue to slog hard in creating a technical spec.

so the question is – whose task was it anyways?

personally, despite the additional burden of having to do the activity of a tda and keeping on hold the ‘testing’ work i am expected to do, i take it as an opportunity to excel at yet another aspect within the dwh life cycle. perhaps a data analyst, data modeler or business analyst role beckons me; but with an organisation as big as the one i work for, should it be down to one tester? shouldnt their be an ownership by the respective tech leads?

should testers be encouraged to get involved in the design stage? or should it be left to someone else? or why shouldn’t they be? after all, dont testers represent the customers and evaluate the product on their behalf?

if testers are encouraged or indeed involved in such areas, will the testing not be more effective? will the test cycles not be shorter (on the whole)? will the initial investment of time in such analysis not help quicker test iterations in the future?

i have a positive outlook on it, despite it requires me, as a tester / test lead to do activities that i’m not quoting for or am not being paid for (perhaps i need ensure that i am), but i’d solicit comments that give me a different perspective too…