It's been 4 amazing years since I started the Saskatoon Testing Discussion Group. I'd tried to start a local interest group a few times before, but didn't have the reach or support needed to be successful. I'd like to thank Vendasta for being a amazing sponsor for the group. They have provided food, drink and space for the meeting for the past four years.
The group has changed a bit over the four years, and some of the sessions were better than others. I think that this latest season has been the best so far. To that end I need to thank the members of the management team that stepped up this year to help me plan and arrange the events. It has made the topics more widely applicable to more of the people in Saskatoon doing test (even on testers doing the role).
We've tried to record and live stream most of our events so that people that can't attend have a way of catching up or attending remotely. We've struggled with the technology for this over the four seasons but it seems to be coming together better lately.
With the end of our latest season drawing near, I wanted to take the time to say thank you to those that come out, those that help and those that have bitten the bullet and spoken at the gatherings. The other thing that I want to do is promote our last session of the season. We have James Bach of Satisfice Inc giving a remote presentation to the group about the layers of testing and how they are all involved in every test. I'm very excited have have James speak to the group. I've heard him talk at more than a few conferences, and he is an excellent presenter that always challenges me to be better at my job. If you want to check him out before the talk, many of talks are on youtube, and is blog can be found here.
Come on out to hear him speak on June 29th at 12:15 (lunch) in the Vendasta Atrium. Pizza and pop provided, Please RSVP at Meetup.com
October 12, 2014
New Season of Discussions at the Saskatoon Software Testing Discussion Group
The new season of discussions is starting this week. The gatherings have moved to Wednesdays from Tuesdays this year. Hopefully this won't be a problem for anyone and actually makes it easier for a few of you to attend. I plan to record the sessions and post them the same as in earlier seasons. They will be available on the you tube channel. In other news, there is now a Facebook group as well as the Google+ and Meetup pages.
The first discussion will be on "Estimating testing effort". I hope all of you locals can make it out.
September 17, 2014
Passion in testing
I've thought a lot about passion and passion in testing lately. I was asked to speak to the september meeting of KWSQA about creating passionate testers. I've found this model for thinking about the training, mentoring and long term careers of testers as a refreshing thought pattern. Passion has some excellent imagery that can relate to careers and to testing.
And so a question for you all.... Think about the things that you are passionate about. Have they changed over the years? Was is sudden? Or a slow change? Were you able to reignite the passion at a later time?
I'm looking forward to talking to many of you about this topic over the next few weeks and months, and I'll let you know how the presentation goes once I'm done.
And so a question for you all.... Think about the things that you are passionate about. Have they changed over the years? Was is sudden? Or a slow change? Were you able to reignite the passion at a later time?
I'm looking forward to talking to many of you about this topic over the next few weeks and months, and I'll let you know how the presentation goes once I'm done.
August 13, 2014
CAST 2014 - DAY 2
There were a lot of great talks today. I was really impressed with all of the Per Scholas graduates that I met today. If you work in testing in the New York Tri-State area and need testers, these men and ladies are top notch. I could talk about the talks that I saw today, and I will. I've decided to do a blog post about each one and have an actionable from each talk that I'll blog about later, but today I just want to talk about the final Keynote.
The Final keynote was by Matt Heusser, and it was about the state of testing. While I don't agree with every thing he said, his desire to inspire us to go out and address the challenges of spreading the work of good testing was evident. There was an important conversation that happened during open season. The audience started to talk about where people in testing were going, and why they were leaving the profession. I understand that this is their experience, but it's not mine.
My experience is that passion is infects. If you haven't met me in person yet, I tend to be very passionate about testing and it comes out when I talk about it. Software Testing is my passion, and my carer. I will talk to anyone about it, and every one. I introduce my self as a Professional Software Tester, and I extol the virtues of software being developed with Professional Software Testers involved from design through to delivery.
I've been a Software Tester and a Test Lead for just over 8 years now. This started after I did two degrees, in Software Engineering and Computer Science, and I found that while I could design and develop, I was much better at Testing. Since that time, I have trained many people to be testers and inspired a few to become testers from other roles. All most all of the people that I have worked with in eight years as testers are still in testing. So I ask myself, why? Why when other are seeing attrition have I seen soo little.
I think it boils down to passion. I work hard, I learn hard and I encourage others to do so. I do it with people at work, I do it with members of my testing community. It's why I lead the Saskatoon Testing Discussion Group. I ensure that testers that work with me have mentors and access to opportunities to learn. If all employees had this let alone testers, there would be happier workplaces.
The Final keynote was by Matt Heusser, and it was about the state of testing. While I don't agree with every thing he said, his desire to inspire us to go out and address the challenges of spreading the work of good testing was evident. There was an important conversation that happened during open season. The audience started to talk about where people in testing were going, and why they were leaving the profession. I understand that this is their experience, but it's not mine.
My experience is that passion is infects. If you haven't met me in person yet, I tend to be very passionate about testing and it comes out when I talk about it. Software Testing is my passion, and my carer. I will talk to anyone about it, and every one. I introduce my self as a Professional Software Tester, and I extol the virtues of software being developed with Professional Software Testers involved from design through to delivery.
I've been a Software Tester and a Test Lead for just over 8 years now. This started after I did two degrees, in Software Engineering and Computer Science, and I found that while I could design and develop, I was much better at Testing. Since that time, I have trained many people to be testers and inspired a few to become testers from other roles. All most all of the people that I have worked with in eight years as testers are still in testing. So I ask myself, why? Why when other are seeing attrition have I seen soo little.
I think it boils down to passion. I work hard, I learn hard and I encourage others to do so. I do it with people at work, I do it with members of my testing community. It's why I lead the Saskatoon Testing Discussion Group. I ensure that testers that work with me have mentors and access to opportunities to learn. If all employees had this let alone testers, there would be happier workplaces.
CAST 2014 - Day 1
What an amazing first day.
It started with a great talk by James Bach about how testing is not Test Cases. Test cases are just an artifact of testing, and much of our work doesn't invlove test cases. This got me to thinking about how what we do is all testing, but very little is test cases, and those test cases are more of a group of ideas for the next tester to think about when they go to test. Even if a tester was to use a test case with detailed steps, the a good professional tester would still go "off script" to better understand what the test case was trying to illumunate.
It was a very full day of learning and talking with other testers. It's been a full day, I'll write more about what I learned later.
August 11, 2014
CAST 2014 - Day 0 (Tutorial Day)
CAST (Conference for the Association for Software Testing) this year is in New York at NYU. I've just spent my first day at the pre-conference tutorials. I spent a really fun day hanging out with other professional testers and learning about how we as people discover things. IE, how do we as people examine things, make guesses as to what they do and refine or refute those guesses until we understand something. It was a really good tutorial session lead by Anne-Marie Charrett. We were looking at research from the social sciences by David Klahr and learning from it so that we could use it to inform how we as testers could improve how we discover.
As testers we discover things all the time. We discover the quality of the item we are testing, we might discover good or bad quality, but the path to discovery is the same. Learning to examine the steps that we are unconsciously taking to make those discoveries allows to to take conscious control over those steps so that they can be refined and improved.
It's been a few full days here in New York, I hope I'll write more later.
August 10, 2014
It's CAST time again
It's that time of year again, and I'm off gallivanting and visiting my favorite testing community at CAST. This year it is in New York at NYU and while the actual conference hasn't started yet, hanging around with this many other professional testers and people that like to think about test is inspiring, motivating and generally a great experience.
I came out early to attend TestRetreatNYC, a open space for test interested people to gather and talk out their testing questions and ideas in wonderful environment. Thanks to LiquidNet and Anna Royzman for letting us use their space, the amazing space certainly helped the discussions. Also thanks to Matt Heusser for organizing it, and Matt Barcomb to helping us run an open space.
I was involved in great discussions on how to grow small testing teams, and testers communities. There were also discussions about the trends in Automated Checking (some call it testing...) but I was at a session about the trials and tribulations of being a lone tester on an agile team. My favorites sessions were about test leadership and the parallels between attending an experiential drama performance and testing.
Like many of these event some of the best discussions happen not in the formal sessions but in the hallways, bars and hotel lobbys where we just throw around ideas and riff off each other. I have some new things to try out, and as before, good or bad, I'll share them here once I've tried.
I came out early to attend TestRetreatNYC, a open space for test interested people to gather and talk out their testing questions and ideas in wonderful environment. Thanks to LiquidNet and Anna Royzman for letting us use their space, the amazing space certainly helped the discussions. Also thanks to Matt Heusser for organizing it, and Matt Barcomb to helping us run an open space.
I was involved in great discussions on how to grow small testing teams, and testers communities. There were also discussions about the trends in Automated Checking (some call it testing...) but I was at a session about the trials and tribulations of being a lone tester on an agile team. My favorites sessions were about test leadership and the parallels between attending an experiential drama performance and testing.
Like many of these event some of the best discussions happen not in the formal sessions but in the hallways, bars and hotel lobbys where we just throw around ideas and riff off each other. I have some new things to try out, and as before, good or bad, I'll share them here once I've tried.
October 8, 2013
Off Week
I was planning to blog about API testing this week, but I seem to have a really busy week this week. The local testing user group is meeting on tuesday and the Saskatoon's First Lean Coffee is this thursday. Currently I lead both groups. So I'll be back next week with API testing. If you have any heuristics or tools that you use for API testing, Please feel free to put them in the comments on this post and I'll include them in next week's series.
October 6, 2013
MindMaps In Testing: Wrap up
So I spent a week without using a notebook, a testing management tool or any notes except for mindmaps. It was an interesting week, I think I actually took better notes, missed fewer dark corners of the applications that I tested and was able to share my work with others easier. Alot if this was made easier by the use of colors on the nodes in my mind maps. I used bold colors for things that I needed to come back to later, and light variants once I had come back and done what ever action was needed.
Lets take a look at what my process looked like. Every Day I reviewed my Maps from earlier in the week to see if there was anything I needed to pull into todays map as a item I planned to do. So before my day even started I had a map with a center node that was the days date, surrounded by any tasks that carried over from the previous days and the items I planned to do that day. If there was a task that carried over the the previous day, I would copy the node over with every thing that was under it so that I had my notes from the earlier work to guide my continuing work. I found that my exploratory testing was very much assisted by this exposure to my previous results. I was able to make links across sessions and generate new charters with better insight than I had when I used a notebook for this same process.
As the day progressed, the first level nodes that I was working on would progress through several states denoted by color. White for upcoming tasks, yellow for the currently active task, Red/Purple for tasks that needed to be revisited, and Green for completed Tasks. If that was all that I had done, this would not be as great a tool as it ended up being. Also made notes on the tasks, and did my test planning under the testing tasks. Since I used an electronic MindMap tool this allowed my test plans to be fluid and dynamic based my results. Results from the tests were also recorded in the MindMap as were all of the defects and other issue I noted. Again these were color coded to allow me to make notes about them and not get distracted by opening my bug reporting tool and spending time then reporting, but rather making enough notes that I could come back later to do a RIMGEA analysis and better report.
I found this technique very useful. My entire day was in one place, I could break pieces out to share and collaborate with others, and I missed less. I will continue to use MindMaps in this way, and will continue to look for new ways to use MindMaps in my work.
Lets take a look at what my process looked like. Every Day I reviewed my Maps from earlier in the week to see if there was anything I needed to pull into todays map as a item I planned to do. So before my day even started I had a map with a center node that was the days date, surrounded by any tasks that carried over from the previous days and the items I planned to do that day. If there was a task that carried over the the previous day, I would copy the node over with every thing that was under it so that I had my notes from the earlier work to guide my continuing work. I found that my exploratory testing was very much assisted by this exposure to my previous results. I was able to make links across sessions and generate new charters with better insight than I had when I used a notebook for this same process.
As the day progressed, the first level nodes that I was working on would progress through several states denoted by color. White for upcoming tasks, yellow for the currently active task, Red/Purple for tasks that needed to be revisited, and Green for completed Tasks. If that was all that I had done, this would not be as great a tool as it ended up being. Also made notes on the tasks, and did my test planning under the testing tasks. Since I used an electronic MindMap tool this allowed my test plans to be fluid and dynamic based my results. Results from the tests were also recorded in the MindMap as were all of the defects and other issue I noted. Again these were color coded to allow me to make notes about them and not get distracted by opening my bug reporting tool and spending time then reporting, but rather making enough notes that I could come back later to do a RIMGEA analysis and better report.
I found this technique very useful. My entire day was in one place, I could break pieces out to share and collaborate with others, and I missed less. I will continue to use MindMaps in this way, and will continue to look for new ways to use MindMaps in my work.
October 4, 2013
Schedule Change
I've realized that my blog schedule isn't working very well. Writing Monday morning seems ok, but I really seem to be writing this post Sunday Night, So I' going to try and Release the Monday post either Sunday Night or Early Monday. The wednesday post doesn't seem to have enough exposure to the tool or technique to have a lot of insight, So I'm going to move this post to Wednesday Night. Same goes for the wrap up post, by writing to for Friday Morning, I'm losing a day of exposure to the topic before I write, so this will be Friday Night/Saturday Morning depending on my availability.
Tune in tomorrow for my MindMaps Wrap-up.
Tune in tomorrow for my MindMaps Wrap-up.
October 2, 2013
SFDIPOT.... Huh?
One thing last weeks exercise showed me was that while I'm familiar with SFDIPOT, I haven't used it enough to be comfortable with it. So let dive in, I'll explain what I know (or think I know) about using it and how I plan to use it this week. If I make mistakes, I fully expect the community to help out and show me where I'm using SFDIPOT wrong, and I'll pass that on to you. The original blog post about this by James Bach on his blog here. He's mentioned that developed this heuristic from surveying six or seven computer science textbooks to lay the framework for his thoughts on application modeling.
As I understand it SFDIPOT is a heuristic to use while doing a software tour to ensure that you have a complete understanding of the application. This is sort of how I use it. I actually do several tours of the application/feature in questions, all very quick and high level while thinking about each of these. I don't do them in the order of the Heuristic simply because I don't really think that way.
So what does it mean?
S: Structure, how is the application put together/wired up. These are things like: Uses Mysql as a server, Has a separate authentication server, Uses memcache and Uses Ajax. Things that can lead you down other vectors that you might not have thought of while doing a simple product tour. You might need to get help/information from the developers and or Documentation for this piece, as such I recommend doing near the end of the heuristic.
F: Function, what are the functions of the feature/application. What are the individual things that it does. This can be a (organized) listing of all of that actions that can be done using the application. These are smaller than the user stories, these are things like "Mark favorite" and "Join mailing list". Stories will come later and will invoke many of these functions.
D: Data, what are the data types involved in the application. Data types can lead to interesting test cases. Have a look at Elisabeth Hendrickson's cheat sheet for some great ideas on how to test your data types once you have identified them.
I: Interfaces, one of the newer letters in this heuristics (and thus harder to get information on), these are the interfaces that the application interacts with the outside world on. Does it have a file based interface anywhere (import/export functions?) Does it store intermediate files to a local storage anywhere? Is an api provided? Does one exists that isn't provided? And yes, the GUI is an interface, does it interact in the ways that a user will expect that kind of interface to interact.
P: Platforms, What are the platforms does this application run on? Is it a java app? Does the version of Java matter? Is it a Web app? Does the browser version or vendor chance the performance or interactions?
O: Operations, What are the operations that the application is trying to accomplish? What is the purpose of this application? What are the mistakes can can happen when trying to do this operation? I've been asked if these are your User Stories and Test Cases, and the answer for me is kind of. User stories are definitely things that are thought of here, but not just the stories, how those stories can go wrong and or get interrupted. Test Cases however are part of your test coverage. I'm using this heuristic to create a PCO which will in turn inform the creation of my test coverage.
T: Timings, Another recent addition to the Heuristic, Timings are what are the time based things you will have to think about with this application? Does your application have to run a specific time? What will happen if it runs on Feb 9th? Or runs over new years Eve? in 1999? What happens when two actions are done too quickly? Too slowly? A recent conversation with also pointed out to me that these can be environmental. Where is the application? What is happening at the same time? Is it rush hour at that time at that location?
As you can see, if you run through all of these, you will have a very extensive list of things to think about while designing your test coverage. As I said in the beginning, this heuristic while seemingly simple does take some practice to use, and that is my goal for the week. To use this heuristic and get better with it. My plan is to use this in my everyday testing, for every feature and bug report from the field that is passed on to me. I realize that this might not align with the it's best uses, but I'm game to try, maybe I'll figure out something new about the heuristic, or about myself.
I'll let you know how it's going on Wednesday!
If I have anything wrong or misinterpreted in this, please let me know in the comments below.
As I understand it SFDIPOT is a heuristic to use while doing a software tour to ensure that you have a complete understanding of the application. This is sort of how I use it. I actually do several tours of the application/feature in questions, all very quick and high level while thinking about each of these. I don't do them in the order of the Heuristic simply because I don't really think that way.
So what does it mean?
S: Structure, how is the application put together/wired up. These are things like: Uses Mysql as a server, Has a separate authentication server, Uses memcache and Uses Ajax. Things that can lead you down other vectors that you might not have thought of while doing a simple product tour. You might need to get help/information from the developers and or Documentation for this piece, as such I recommend doing near the end of the heuristic.
F: Function, what are the functions of the feature/application. What are the individual things that it does. This can be a (organized) listing of all of that actions that can be done using the application. These are smaller than the user stories, these are things like "Mark favorite" and "Join mailing list". Stories will come later and will invoke many of these functions.
D: Data, what are the data types involved in the application. Data types can lead to interesting test cases. Have a look at Elisabeth Hendrickson's cheat sheet for some great ideas on how to test your data types once you have identified them.
I: Interfaces, one of the newer letters in this heuristics (and thus harder to get information on), these are the interfaces that the application interacts with the outside world on. Does it have a file based interface anywhere (import/export functions?) Does it store intermediate files to a local storage anywhere? Is an api provided? Does one exists that isn't provided? And yes, the GUI is an interface, does it interact in the ways that a user will expect that kind of interface to interact.
P: Platforms, What are the platforms does this application run on? Is it a java app? Does the version of Java matter? Is it a Web app? Does the browser version or vendor chance the performance or interactions?
O: Operations, What are the operations that the application is trying to accomplish? What is the purpose of this application? What are the mistakes can can happen when trying to do this operation? I've been asked if these are your User Stories and Test Cases, and the answer for me is kind of. User stories are definitely things that are thought of here, but not just the stories, how those stories can go wrong and or get interrupted. Test Cases however are part of your test coverage. I'm using this heuristic to create a PCO which will in turn inform the creation of my test coverage.
T: Timings, Another recent addition to the Heuristic, Timings are what are the time based things you will have to think about with this application? Does your application have to run a specific time? What will happen if it runs on Feb 9th? Or runs over new years Eve? in 1999? What happens when two actions are done too quickly? Too slowly? A recent conversation with also pointed out to me that these can be environmental. Where is the application? What is happening at the same time? Is it rush hour at that time at that location?
As you can see, if you run through all of these, you will have a very extensive list of things to think about while designing your test coverage. As I said in the beginning, this heuristic while seemingly simple does take some practice to use, and that is my goal for the week. To use this heuristic and get better with it. My plan is to use this in my everyday testing, for every feature and bug report from the field that is passed on to me. I realize that this might not align with the it's best uses, but I'm game to try, maybe I'll figure out something new about the heuristic, or about myself.
I'll let you know how it's going on Wednesday!
If I have anything wrong or misinterpreted in this, please let me know in the comments below.
MindMaps in Testing: Mid-week
A few days in of increased use of MindMaps in my daily routine and I have seen some interesting results. Like any technique that is new, the novelty of doing something a new way can drive higher short term adoption, and I have been able to consistently use my day notes mind map to track my day. I would normally try to write this in a log book, but I tend to forget. The ability to color code the nodes however has made this more likely to continue after the "New way" interest has worn off.
So what have I done with mind maps so far? Two of my common tasks have been moved over to mind maps. Test planning for newly completed features, and my daily activity log. I have in fact combined these two into a single map, one for each day, where testing won't span more than a day. Where testing will be a multi-day or multi tester effort, I've split it out into a separate map. The daily activity map has helped me track my day and make notes on the activities that I do. I've marked with color finished tasks, blocked task, bugs found, future investigations and test results. The color enables me to quickly look and see what I need to remember. I used to do this with my notebook, but it's been hard to find the tasks and future investigations easily later when I go looking.
The tool that I've been using has a mode where it will auto color the nodes for you based on a color change at the leaf level to indicate that you have completed a task or been blocked if you are using your MindMap as a plan/to-do. I don't usually write detailed plans, I write down a list of areas I want to test, and some notes under each of those about why or how of that testing area. This feature the MindMap software that I'm using allows me to quickly look back and see what area's I have left to cover in upcoming charters, and update based on the charters that I've completed.
All in all, MindMaps have been a good addition to my work. I haven't had to share the information in them with others yet, but I expect that to be helpful also.
So what have I done with mind maps so far? Two of my common tasks have been moved over to mind maps. Test planning for newly completed features, and my daily activity log. I have in fact combined these two into a single map, one for each day, where testing won't span more than a day. Where testing will be a multi-day or multi tester effort, I've split it out into a separate map. The daily activity map has helped me track my day and make notes on the activities that I do. I've marked with color finished tasks, blocked task, bugs found, future investigations and test results. The color enables me to quickly look and see what I need to remember. I used to do this with my notebook, but it's been hard to find the tasks and future investigations easily later when I go looking.
The tool that I've been using has a mode where it will auto color the nodes for you based on a color change at the leaf level to indicate that you have completed a task or been blocked if you are using your MindMap as a plan/to-do. I don't usually write detailed plans, I write down a list of areas I want to test, and some notes under each of those about why or how of that testing area. This feature the MindMap software that I'm using allows me to quickly look back and see what area's I have left to cover in upcoming charters, and update based on the charters that I've completed.
All in all, MindMaps have been a good addition to my work. I haven't had to share the information in them with others yet, but I expect that to be helpful also.
September 30, 2013
Mind Maps as a Tool for Testing
A lot of people have written about mind map and testing recently because the tools for working with and sharing them have gotten so much better. There are really smart excellent testers out there that use mind maps in their daily work flow, and I've been trying to add them to mine since I got back from CAST 2013. This week I'll use them any time I would have written anything down.
For example:
For example:
- My activity log for the day complete with bug and investigations will be a mind map.
- Testing Plans
- Exploratory Charters and the notes/results from them
- Feature/Bug/Product outlines will also be done up this way.
I hope to create 4 or more Mind maps a day for the next 5 days. It should be an interesting experience.
Read on for some dryer text on what mind maps are and how I will be doing them.
September 27, 2013
Random Data Tool: Wrap up
A week of using a random data tool has not shown the amazing benefit I hoped it would. This might have to do with the tool that I choose to use, and might have been that I wasn't really using the tool the way it was designed.
I prefer to use self-validating data as often as I can, and thus random data isn't always useful. When it was useful, I used it, but ran into a few blocks. The service that I was using didn't provide all the types that I was wanting (valid US phone numbers, multiple formats), or didn't provide enough variation to suit me for the data types it did have (Like name, only used normal english names) I was hoping that this tool would help drive me into some of the corners of the data set, but all it really did was keep me in the center of the path with different values that didn't diverge far from the norm.
I prefer to use self-validating data as often as I can, and thus random data isn't always useful. When it was useful, I used it, but ran into a few blocks. The service that I was using didn't provide all the types that I was wanting (valid US phone numbers, multiple formats), or didn't provide enough variation to suit me for the data types it did have (Like name, only used normal english names) I was hoping that this tool would help drive me into some of the corners of the data set, but all it really did was keep me in the center of the path with different values that didn't diverge far from the norm.
September 24, 2013
Random Data tool
I was originally going to write about a API testing heuristic that I stumbled upon recently for this weeks blog series, but I'm not planning on doing any APi testing this week. So instead I'm going to write about a new tool/website that I recently discovered for generating random data of various types. TestSpicer provides a set of post/get api calls that will generate "Random" data for you. Need users name for a test account? Call the name endpoint. Need an image? There's a call for that. In Fact there are currently 19 different end points, many I won't use, but this week, I'll try a few and see if it helps me find some a bug by creating an account that I wouldn't.
September 23, 2013
RIMGEA: Wrap-up
Using RIMGEA for the past week has been a great exercise. It certainly improved my bug reports. I didn't get to the point of all my bug reports being fully RIMGEA or the bugs fixes that I reviewed being fully explored using RIMGEA, but I did see a improvement in my bug reports and bug exploration.
Lets look at these two sides of how I used RIMGEA and the challenges that caused me to shortcut using it fully.
RIMGEA as a bug reporting Heuristic:
This is the intended use for this Heuristic, and it works well here. If you are rushed for time, or have a large number of bugs to report, it's unlikely that you will use all of it. For some bugs I didn't get past the R(eplicate). Once I got my replication steps, I could write a convincing bug report. However, this past week I tried to get further with every bug. My reports were definitely better. Reducing all the possible variables out for the I(solate) was hard, and often I overlooked some, but looking for them certainly helped my testing in other areas. I think I'll do this exercise every few weeks until I can consistently get into the last few letters of the Heuristic.
RIMGEA as a bug verification Heuristic:
This was a bit of an off-label use, but I thought that if the bug hadn't had this done to it when it was written, the fix might not cover all of the incarnations of the bug either. It worked for me. Often replicating the bug in the pre-patched system showed the the bug had been fixed earlier and the new patch wasn't actually needed, or that there were other paths into the bug that had not been examined and fixed.
All in all, this was a productive exercise. I'll do this one again, and share it with my team.
Lets look at these two sides of how I used RIMGEA and the challenges that caused me to shortcut using it fully.
RIMGEA as a bug reporting Heuristic:
This is the intended use for this Heuristic, and it works well here. If you are rushed for time, or have a large number of bugs to report, it's unlikely that you will use all of it. For some bugs I didn't get past the R(eplicate). Once I got my replication steps, I could write a convincing bug report. However, this past week I tried to get further with every bug. My reports were definitely better. Reducing all the possible variables out for the I(solate) was hard, and often I overlooked some, but looking for them certainly helped my testing in other areas. I think I'll do this exercise every few weeks until I can consistently get into the last few letters of the Heuristic.
RIMGEA as a bug verification Heuristic:
This was a bit of an off-label use, but I thought that if the bug hadn't had this done to it when it was written, the fix might not cover all of the incarnations of the bug either. It worked for me. Often replicating the bug in the pre-patched system showed the the bug had been fixed earlier and the new patch wasn't actually needed, or that there were other paths into the bug that had not been examined and fixed.
All in all, this was a productive exercise. I'll do this one again, and share it with my team.
September 18, 2013
RIMGEA: Mid week
This one has been a little easier to do. I'm not trying to use it outside of its scope, and that has helped. Additionally, I know that a well written bug report helps the developers find and fix the bug, but sometimes I really don't need as much information on a bug as RIMGEA would have me provide. Nor I do not have the time to do all of RIMGEA for the six bugs that I need to file before I leave work for the day. That said, having a note on my desk reminding me to use it has increased the quality of the bug report, just not all the way.
One change that I didn't expect was the change in how I've been verifying bug fixes. Many of these bugs come on from external sources or non testers. Many get to the developers before a testers has had a look at them. Thus when they do get to me, they rarely have replication steps let alone any of the other items outlined by RIMGEA. Filling in a very quick RIMGEA has increased the depth at which I verify these bug fixes to almost a full exploratory testing session. Making the bug fix verifications more through. It's caught a minor issue that I might not have caught otherwise.
So Far So Good with this Heuristic. More on Friday!
One change that I didn't expect was the change in how I've been verifying bug fixes. Many of these bugs come on from external sources or non testers. Many get to the developers before a testers has had a look at them. Thus when they do get to me, they rarely have replication steps let alone any of the other items outlined by RIMGEA. Filling in a very quick RIMGEA has increased the depth at which I verify these bug fixes to almost a full exploratory testing session. Making the bug fix verifications more through. It's caught a minor issue that I might not have caught otherwise.
So Far So Good with this Heuristic. More on Friday!
September 16, 2013
RIMGEA: A Bug reporting Heuristic
Last week I looked at a heuristic that helped me with testing, and planning my testing. It even helped me with isolating a bug and replicating it. If that was all that I did, it might be the only heuristic that I need in my daily work. (Un?)Fortunately I also find bugs, and bug reports are one of the main forms of written communication that exists for me or my team. Thus, a well written bug report that has all of the elements can be important, or at least as important as the bug we are reporting.
This week I'm going to concentrate on writing better bug reports by using the RIMGEA heuristic. This heuristic developed by Cem Kaner and gives the user a series of steps to go through to ensure that a bug report has enough information that others can correctly evaluate and prioritize the bug you are reporting.
R, Replicate: Ideally all bugs that we report are replicable, we might not understand all the variables involved in the bug yet, but in order to move on in this heuristic the bug needs to have a series of steps that will replicate the bug. If it doesn't we need to spend some time coming up with some steps. If these steps only sometimes replicate the bug, try to refine them so that it can consistently replicate the bug or at least describe how often these steps can replicate the bug. Sometimes how often a bug can be replicated using the same steps is an important piece of information.
I, Isolate: Any given set of replication steps will have any number of things that just happen to be that way, but the bug doesn't need them. Identify these elements so that we know what is really involved in the bug. This step will help with the next two.
M, Maximize: This is to find out just how bad this could be if everything went exactly wrong. The version of the bug that you have discovered or have been asked to improve the report for is not likely the worst case version. Since you have isolated the bug you should be able to determine how to make the effect greater.
G, Generalize: This goes part and parcel with Maximize and Isolate. Since we know what the isolation is and how to maximize it, the other side of this coin is what is the easiest version of the steps, or what is the most common path a user could follow to trigger this bug.
E, Externalize: This one is important, not to the bug, but to your ability to inform others of the effect of the bug. To externalize, you describe the impact of the bug to the users, their data and their ability to complete the task that they are trying to complete with this application.
And blandize: Ok, they stretched it on this one to make it an A, but it is an important aspect of any bug report. To find a way to report your bug without making it personal. They could have used O for objective there also and it might have made it clearer to the users
Interestingly, I filed a bug report for work while writing this blog post. My bug reports will have to change a lot to this week. It's not that my bug reports are incomplete, I feel that I write them to the appropriate level of detail for who will need to read them. However, I didn't do some of these steps. It should be interesting.
This week I'm going to concentrate on writing better bug reports by using the RIMGEA heuristic. This heuristic developed by Cem Kaner and gives the user a series of steps to go through to ensure that a bug report has enough information that others can correctly evaluate and prioritize the bug you are reporting.
R, Replicate: Ideally all bugs that we report are replicable, we might not understand all the variables involved in the bug yet, but in order to move on in this heuristic the bug needs to have a series of steps that will replicate the bug. If it doesn't we need to spend some time coming up with some steps. If these steps only sometimes replicate the bug, try to refine them so that it can consistently replicate the bug or at least describe how often these steps can replicate the bug. Sometimes how often a bug can be replicated using the same steps is an important piece of information.
I, Isolate: Any given set of replication steps will have any number of things that just happen to be that way, but the bug doesn't need them. Identify these elements so that we know what is really involved in the bug. This step will help with the next two.
M, Maximize: This is to find out just how bad this could be if everything went exactly wrong. The version of the bug that you have discovered or have been asked to improve the report for is not likely the worst case version. Since you have isolated the bug you should be able to determine how to make the effect greater.
G, Generalize: This goes part and parcel with Maximize and Isolate. Since we know what the isolation is and how to maximize it, the other side of this coin is what is the easiest version of the steps, or what is the most common path a user could follow to trigger this bug.
E, Externalize: This one is important, not to the bug, but to your ability to inform others of the effect of the bug. To externalize, you describe the impact of the bug to the users, their data and their ability to complete the task that they are trying to complete with this application.
And blandize: Ok, they stretched it on this one to make it an A, but it is an important aspect of any bug report. To find a way to report your bug without making it personal. They could have used O for objective there also and it might have made it clearer to the users
Interestingly, I filed a bug report for work while writing this blog post. My bug reports will have to change a lot to this week. It's not that my bug reports are incomplete, I feel that I write them to the appropriate level of detail for who will need to read them. However, I didn't do some of these steps. It should be interesting.
September 13, 2013
SFDIPOT: end of week Review
Well it's been a week of concentrating on using SFDIPOT in my testing activities. So did it make me a superstar tester? Did it show me every bug? Was my team amazed at the testing plans I wrote? Could it cure cancer and bring about world peace? These are all (well mostly) important questions that I have to answer when I ask myself the most important questions: Will I keep using it? Will I encourage others to use it?
Did it make me a superstar tester? No, but it did have some excellent benefits. It reduced my testing time be giving me better direction when I setup my testing plans. It gave me better insight into bug investigations. It gave me new ideas of areas to test to add to my list already too long list of areas to investigate. I wasn't really looking for a single item to change and I'd become a superstar, I was looking to improve, and learn. That part of my mission was a success.
Did it show me every bug? No, but I was able to find some sooner than I would have otherwise. I don't think there is any tool or technique out there that will find every bug, the search space to find every bug is impossibly large. So a better metric might be did it help me find more bugs, or find the bugs faster? In my experience from this past week, yes it did.
Was my team amazed with my new testing plans? Not really, for me, it didn't change much. I can see it creating better testing plans for some, but in my environment, working in a agile development team, I don't often share my testing plans. My testing plans are usually something I write for myself so that I don't forget anything. Being a single tester on a team has lead to some bad habits, but I'll deal with those another week.
In all it's been a good week. I enjoyed this emphasis on SFDIPOT, and the extra thought it helped me do in my testing activities. I didn't really use it as much as I meant to, I'll keep using and teaching it to the team of testers where I work, and I'll keep you updated if I learn anything new about it in that journey.
Did it make me a superstar tester? No, but it did have some excellent benefits. It reduced my testing time be giving me better direction when I setup my testing plans. It gave me better insight into bug investigations. It gave me new ideas of areas to test to add to my list already too long list of areas to investigate. I wasn't really looking for a single item to change and I'd become a superstar, I was looking to improve, and learn. That part of my mission was a success.
Did it show me every bug? No, but I was able to find some sooner than I would have otherwise. I don't think there is any tool or technique out there that will find every bug, the search space to find every bug is impossibly large. So a better metric might be did it help me find more bugs, or find the bugs faster? In my experience from this past week, yes it did.
Was my team amazed with my new testing plans? Not really, for me, it didn't change much. I can see it creating better testing plans for some, but in my environment, working in a agile development team, I don't often share my testing plans. My testing plans are usually something I write for myself so that I don't forget anything. Being a single tester on a team has lead to some bad habits, but I'll deal with those another week.
In all it's been a good week. I enjoyed this emphasis on SFDIPOT, and the extra thought it helped me do in my testing activities. I didn't really use it as much as I meant to, I'll keep using and teaching it to the team of testers where I work, and I'll keep you updated if I learn anything new about it in that journey.
September 11, 2013
Mid-Week with SFDIPOT
Its been two days of my renewed attempt of learning SFDIPOT by using it for all of my testing activities. So far I have used it in conjunction with a product tour, a bug verification/investigation, and a the test coverage planning for a single feature. I've been surprised to find that it helped for all three. The bug investigation was the one that surprised me. I thought I might have to heavily modify SFDIPOT to use it, but it has started to flow nicely and more comfortably. Here's some quick thoughts. from the early week.
Understanding a technique takes time, and often trying to explain it to others. I think I learned the most from about SFDIPOT writing monday's blog post. Michael Bolton and James Bach were both generous with their time and knowledge and review my post before it went live and discussed with me some the place where I wasn't truly relaying what they meant. There are still a few places in my post they are not completely what they mean, but they are in areas where I'm still not sure what they mean.
There's never too small a item to use this technique if it is stumping you. Yesterday I had a bug report come in from the field. The replication steps were simple, except they didn't expose the bug except at the customer site. After spending some time trying to figure out what the difference was I pulled out SFDIPOT and took the time to do a structured approach to my thinking about my investigation. Very quickly I identified the missing variable and was able to get refined replication steps that reliably reproduced the bug.
The last thing this exercise has shown me is the the structured approach helps me even when I'm working with a old existing feature that I've tested many times. Today I was able to quickly identify a place where testability could be improved while i was doing a code review and using SFDIPOT to guide my thoughts. I haven't even thought of the test coverage yet, but I already know it will be easier to test because I was able to fully think about the feature.
It's going well so far, I've even had some developers asking about it! I'll be back on friday with the round up!
Understanding a technique takes time, and often trying to explain it to others. I think I learned the most from about SFDIPOT writing monday's blog post. Michael Bolton and James Bach were both generous with their time and knowledge and review my post before it went live and discussed with me some the place where I wasn't truly relaying what they meant. There are still a few places in my post they are not completely what they mean, but they are in areas where I'm still not sure what they mean.
There's never too small a item to use this technique if it is stumping you. Yesterday I had a bug report come in from the field. The replication steps were simple, except they didn't expose the bug except at the customer site. After spending some time trying to figure out what the difference was I pulled out SFDIPOT and took the time to do a structured approach to my thinking about my investigation. Very quickly I identified the missing variable and was able to get refined replication steps that reliably reproduced the bug.
The last thing this exercise has shown me is the the structured approach helps me even when I'm working with a old existing feature that I've tested many times. Today I was able to quickly identify a place where testability could be improved while i was doing a code review and using SFDIPOT to guide my thoughts. I haven't even thought of the test coverage yet, but I already know it will be easier to test because I was able to fully think about the feature.
It's going well so far, I've even had some developers asking about it! I'll be back on friday with the round up!
Subscribe to:
Posts (Atom)