![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
Friday, November 16th, 2012 12:48 am
work, rewind and then fast forward, in progress
Mostly for my own sanity, the reason why I ended up ordering new boots from Amazon in a fit of retail therapy due to work. They are goddamn awesome boots, and just wearing them made me feel better as a human being, so.
In this entry, I said that two weeks of work was basically scrubbed. That's inaccurate.
They scrubbed a total of 1.5 months of total testing for this build. The two weeks refers to the UAT testing period. This is a short overview of a build to clarify why this is stroke-inducing.
Keywords:
Mods - Modifications - in general, these are new things added to the system as opposed to fixes to code already in the system. Example: adding new sections to the online application, adding new functions like upload to the online application. Also can cover improvements that make something work better. This is brand new or newly modified code. It does not fix anything that already exists.
Maintenance - in general, these are fixes to existing code, cosmetic alterations, spelling errors.
End to End - this is a type of test that integrates all the separate programs together in a single test. We start working as clients filling out an online benefits application, switch to clerks to claim the task and schedule in Portal (STP), then switch to caseworkers to work the case in TIERS and check everything that a caseworker or client woudl need/do including correspondence, benefit amounts, and well, all of it. These can be forty steps to eighty steps and are for the most part take a whole day to complete.
The Anatomy of a Build
If the build is XX.0 - Big Build, contains major modifications and additions to the programming as well as fixes and maintenance on existing items. It contains three distinct testing periods as well as three Regression testing periods. If a build is XX.2, it has a very small amount of a and all of c below.
a.) System Testing (ST) - two weeks to a month - done in the SIT1 or SIT2 environments - minimal testing of stripped down scripts to confirm the new code has been added to the environment, the right code was added to the environment, and to shake out really huge problems like the code breaks on the first line or someething. System Testing only applies to modifications, not maintenance. All of the above happened so regularly during regular testing that this testing period was specifically added. Yes, we have had times that they simply thought they deployed the code and didn't or deployed it wrong during UAT testing, which is very, very bad.
b.) Joint SIT Testing (Joint SIT) - done in the SIT1 or SIT2 environments - short version--we test several programs in four separate environments for the purpose of this explanation. ST and Joint SIT are done in the SIT environments. SIT is done by testers who work for our vendor who also does the coding and development of the program, so separate group of testers that test one build ahead of us. However, for mods, in this testing period, they test with us on select mods that require that the three major programs used by casewrokers and clients all actually interact with each other correctly.
This is where it gets weird and doesn't make sense, but just go with it. In the SIT environment, they only own--in a matter of speaking--two of the three programs, while we own the other one. I don't know what this means, except that during this particular testing, we switch off with them running end to end tests in sections. They're shorter than end to ends, but longer than single program tests. I also really don't know why we do this, but it is apparently hugely important.
This starts after a week or so of system testing and can continue until UAT testing starts. So one week to a month.
c.) UAT Testing - done in the UAT1 and UAT2 and lasts roughly four weeks and includes the following.
i.) Acceptance Testing - -UAT1 or UAT2 environment - this usually takes one day. This is basically very simple regression testing with short tests that test what the system was doing before this modification. An example would be running a test to make sure you can still schedule an appointment. We usually get three to five each and are generally related to the modifications in some way, but usually are not being changed in the current build.
ii.) UAT - User Acceptance Testing - UAT1 or UAT2 environment, sometimes both - this is where the magic happens. ST and Joint SIT as well as the much earlier SIT testing by their testers should in theory make this easy and with few or no defects. The bulk of our defects still come from this period. Problems with the program not following state or federal policy, random environmental errors, code breaks that only happen when you ask for the most common program people apply for, obscure code breaks that only happen during the full moon, and everything in between. In the last eighteen months, we've had no less than three delayed deployments come out of what happened during this testing period and not all the defects were fixed. In one build, half of the ones for one mod could not be fixed before deployment.
iii.) Regression - UAT1 or UAT2 environment - as soon as you personally finish all your UAT tests, you start Regression, which is a combination of Acceptance--things that were working at teh beginning of teh build but due to defects and code pushing, God knows--and UAT tests that have been added in that cover maintenance items and sometimes mods. This can take from three days to a week to finish depending on how many. We get defects out of here every time.
d.) Prodfix - done in Prodfix - this environment is the only one that is in real time and not aging, uses the most recent copy of the entire database, and is teh cleaned up version of the code that will be deployed to production. It is basically a clone of what will occur after deployment.
i.) Acceptance testing - identical to UAT testing but usually lasts one day and a tester is assigned one or two tests.
ii.) Prodfix testing - each tester will submit one to three tests from each modification they scripted and we have three days to test all of them. We get defects out of this every time. These have to be finished by the third day because deployment is the next day. There are no exceptions to this and God help you if you get a serious defect now.
Afterword
So now that you have that, to clarify; when i say we lost two weeks of testing, I mean UAT testing. We also lost all of ST and Joint SIT, so about a month and a half of testing is scrubbed because they removed the Oracle upgrade for Reasons. As that runs the database, all of our tests are basically nulled and have to be re-run.
To facilitate this, this build won't go out at its' normal deployment time but will be delayed to go out simultaneously with the next deployment, which is a XX.2, a mostly-maintenance build, which is scheduled for February. That is almost good news, but the problem is, there is a math of linear time problem that seems to come up when things like this occur. As it looks like we have more time, even though it's the same amount of time we would usually have for two builds, XX.2 is being overloaded with more items for us to cover, because magic. This is in addition to a special yearly build we do for just COLA (cost of living) updates that have to be analyzed, scripted, tested, and without defects at all before the end of the year. The work of two builds plus a special build plus additional items because magic time will be done in a period of time that is effectively the amount of time we would normally spend completing one XX.0 and looking at the next XX.2 build.
And just to make this much more surreal, this particular XX.0 build was already shortened by a week due to Thanksgiving, so--I have no idea.
Personally in my tests, one of the BRDs--that's the written blueprint of what a modification does--has to be clarified and rewritten to accommodate a conflict with a build that occurred a year ago and no one noticed, and so I will have to rewrite some of the tests, but CYA and test auditing by auditors mean I have to argue that the BRD as written is, while very technically correct, also wrong and needs to be rewritten, because it lacks clarification on a particular very messy point that makes my rewritten tests both technically correct but to anyone reading, also make no sense as to why I'm doing it without an explanation the size of War and Peace and references to former builds that I didn't test those things in.
Personally in my tests, one of my assigned mods has yet to get the code for it to even be tested. Which at this point doesn't even matter, but it's annoying.
This is why at work when my defects are put in withdrawal and my gentle explanation of why I am right goes ignored or my questions as to why are not explained, my lead or someone on my team officially reads my emails or defects for sarcasm or potential condescension before I send them out, though warfare through handbook quoting is exempt from this rule.
(I recently corrupted one of the developers to the point where when he was discussing one of the defects with my manager (that he agreed with), my manager noted his phrasing was really familiar and stared at me meaningfully. Which I think means he's awesome.)
The last time that I was unchecked during a chain mail argument that lasted several fast-paced email hours, I showed Madelyn the email chain, where she noted I had unironically described a button as 'limned', 'glowy', and various synonyms of ridiculous, pointless, and aesthetically offensive, but with nicer words, more than once, and sometimes at a length more appropriate to short stories. On the other hand, my manager did agree that for me, it was restrained and surprisingly free of an undercurrent of despair for humanity, so you know, there's that. And like, how many times in your life can you use the phrase 'yellow-limned', 'glowy', and 'non-functional' all in the same sentence? Not many.
I also have been forbidden to create any kind of macro to illustrate my feelings, including puppies or kittens looking sad in the rain or boats that seem to be sinking tragically. And I cannot title any email with a reference to Star Wars, Star Trek, or anything that implies I am a Jedi working for Sith, a desperate individual trying and failing to escape the Borg, or that the force has abandoned us to die. As apparently, one forwarded email that made the rounds was titled Attack of the Clones Server Error (as it was a literal clone server error), and non-geeks in management were very confused. Which is just the saddest thing I have ever heard.
However, on a lighter note, because embarrassment never gets old, there was an unexpected meeting in the room beside the break room the day I downloaded "Give Your Heart a Break". As that area is usually deserted, I had no qualms (quietly!) singing along (quietly, I swear) while doing something that might be considered a skip-hop-turn-slide-bounce past the break room, by the meeting, and down an empty cubicle row, perhaps several times. I was wearing headphones, which is my only defense for how I missed it, but apparently the end of the meeting was getting grumpy due to it going overtime and then the wobbling refrain drifted through the room by way of the half-open door and everyone came to the door to watch me.
Later that afternoon, amid a disturbing number of random smiles in my direction by people I did not know wandering around our area, I was informed of how much my performance was enjoyed. And my manager requested "Glad You Came" if I felt like doing a repeat.
Now I wonder every time I send an email if the first thing they think of when they see my name is the cubicle shuffle of pop music. And by that I mean, they totally do.
My life.
In case anyone is curious, I went for an eye exam earlier today and during the process there was a period of intense light per eye that has since developed into a headache that eased off just in time for me to reach terminal insomnia. This is fun for me.
In this entry, I said that two weeks of work was basically scrubbed. That's inaccurate.
They scrubbed a total of 1.5 months of total testing for this build. The two weeks refers to the UAT testing period. This is a short overview of a build to clarify why this is stroke-inducing.
Keywords:
Mods - Modifications - in general, these are new things added to the system as opposed to fixes to code already in the system. Example: adding new sections to the online application, adding new functions like upload to the online application. Also can cover improvements that make something work better. This is brand new or newly modified code. It does not fix anything that already exists.
Maintenance - in general, these are fixes to existing code, cosmetic alterations, spelling errors.
End to End - this is a type of test that integrates all the separate programs together in a single test. We start working as clients filling out an online benefits application, switch to clerks to claim the task and schedule in Portal (STP), then switch to caseworkers to work the case in TIERS and check everything that a caseworker or client woudl need/do including correspondence, benefit amounts, and well, all of it. These can be forty steps to eighty steps and are for the most part take a whole day to complete.
The Anatomy of a Build
If the build is XX.0 - Big Build, contains major modifications and additions to the programming as well as fixes and maintenance on existing items. It contains three distinct testing periods as well as three Regression testing periods. If a build is XX.2, it has a very small amount of a and all of c below.
a.) System Testing (ST) - two weeks to a month - done in the SIT1 or SIT2 environments - minimal testing of stripped down scripts to confirm the new code has been added to the environment, the right code was added to the environment, and to shake out really huge problems like the code breaks on the first line or someething. System Testing only applies to modifications, not maintenance. All of the above happened so regularly during regular testing that this testing period was specifically added. Yes, we have had times that they simply thought they deployed the code and didn't or deployed it wrong during UAT testing, which is very, very bad.
b.) Joint SIT Testing (Joint SIT) - done in the SIT1 or SIT2 environments - short version--we test several programs in four separate environments for the purpose of this explanation. ST and Joint SIT are done in the SIT environments. SIT is done by testers who work for our vendor who also does the coding and development of the program, so separate group of testers that test one build ahead of us. However, for mods, in this testing period, they test with us on select mods that require that the three major programs used by casewrokers and clients all actually interact with each other correctly.
This is where it gets weird and doesn't make sense, but just go with it. In the SIT environment, they only own--in a matter of speaking--two of the three programs, while we own the other one. I don't know what this means, except that during this particular testing, we switch off with them running end to end tests in sections. They're shorter than end to ends, but longer than single program tests. I also really don't know why we do this, but it is apparently hugely important.
This starts after a week or so of system testing and can continue until UAT testing starts. So one week to a month.
c.) UAT Testing - done in the UAT1 and UAT2 and lasts roughly four weeks and includes the following.
i.) Acceptance Testing - -UAT1 or UAT2 environment - this usually takes one day. This is basically very simple regression testing with short tests that test what the system was doing before this modification. An example would be running a test to make sure you can still schedule an appointment. We usually get three to five each and are generally related to the modifications in some way, but usually are not being changed in the current build.
ii.) UAT - User Acceptance Testing - UAT1 or UAT2 environment, sometimes both - this is where the magic happens. ST and Joint SIT as well as the much earlier SIT testing by their testers should in theory make this easy and with few or no defects. The bulk of our defects still come from this period. Problems with the program not following state or federal policy, random environmental errors, code breaks that only happen when you ask for the most common program people apply for, obscure code breaks that only happen during the full moon, and everything in between. In the last eighteen months, we've had no less than three delayed deployments come out of what happened during this testing period and not all the defects were fixed. In one build, half of the ones for one mod could not be fixed before deployment.
iii.) Regression - UAT1 or UAT2 environment - as soon as you personally finish all your UAT tests, you start Regression, which is a combination of Acceptance--things that were working at teh beginning of teh build but due to defects and code pushing, God knows--and UAT tests that have been added in that cover maintenance items and sometimes mods. This can take from three days to a week to finish depending on how many. We get defects out of here every time.
d.) Prodfix - done in Prodfix - this environment is the only one that is in real time and not aging, uses the most recent copy of the entire database, and is teh cleaned up version of the code that will be deployed to production. It is basically a clone of what will occur after deployment.
i.) Acceptance testing - identical to UAT testing but usually lasts one day and a tester is assigned one or two tests.
ii.) Prodfix testing - each tester will submit one to three tests from each modification they scripted and we have three days to test all of them. We get defects out of this every time. These have to be finished by the third day because deployment is the next day. There are no exceptions to this and God help you if you get a serious defect now.
Afterword
So now that you have that, to clarify; when i say we lost two weeks of testing, I mean UAT testing. We also lost all of ST and Joint SIT, so about a month and a half of testing is scrubbed because they removed the Oracle upgrade for Reasons. As that runs the database, all of our tests are basically nulled and have to be re-run.
To facilitate this, this build won't go out at its' normal deployment time but will be delayed to go out simultaneously with the next deployment, which is a XX.2, a mostly-maintenance build, which is scheduled for February. That is almost good news, but the problem is, there is a math of linear time problem that seems to come up when things like this occur. As it looks like we have more time, even though it's the same amount of time we would usually have for two builds, XX.2 is being overloaded with more items for us to cover, because magic. This is in addition to a special yearly build we do for just COLA (cost of living) updates that have to be analyzed, scripted, tested, and without defects at all before the end of the year. The work of two builds plus a special build plus additional items because magic time will be done in a period of time that is effectively the amount of time we would normally spend completing one XX.0 and looking at the next XX.2 build.
And just to make this much more surreal, this particular XX.0 build was already shortened by a week due to Thanksgiving, so--I have no idea.
Personally in my tests, one of the BRDs--that's the written blueprint of what a modification does--has to be clarified and rewritten to accommodate a conflict with a build that occurred a year ago and no one noticed, and so I will have to rewrite some of the tests, but CYA and test auditing by auditors mean I have to argue that the BRD as written is, while very technically correct, also wrong and needs to be rewritten, because it lacks clarification on a particular very messy point that makes my rewritten tests both technically correct but to anyone reading, also make no sense as to why I'm doing it without an explanation the size of War and Peace and references to former builds that I didn't test those things in.
Personally in my tests, one of my assigned mods has yet to get the code for it to even be tested. Which at this point doesn't even matter, but it's annoying.
This is why at work when my defects are put in withdrawal and my gentle explanation of why I am right goes ignored or my questions as to why are not explained, my lead or someone on my team officially reads my emails or defects for sarcasm or potential condescension before I send them out, though warfare through handbook quoting is exempt from this rule.
(I recently corrupted one of the developers to the point where when he was discussing one of the defects with my manager (that he agreed with), my manager noted his phrasing was really familiar and stared at me meaningfully. Which I think means he's awesome.)
The last time that I was unchecked during a chain mail argument that lasted several fast-paced email hours, I showed Madelyn the email chain, where she noted I had unironically described a button as 'limned', 'glowy', and various synonyms of ridiculous, pointless, and aesthetically offensive, but with nicer words, more than once, and sometimes at a length more appropriate to short stories. On the other hand, my manager did agree that for me, it was restrained and surprisingly free of an undercurrent of despair for humanity, so you know, there's that. And like, how many times in your life can you use the phrase 'yellow-limned', 'glowy', and 'non-functional' all in the same sentence? Not many.
I also have been forbidden to create any kind of macro to illustrate my feelings, including puppies or kittens looking sad in the rain or boats that seem to be sinking tragically. And I cannot title any email with a reference to Star Wars, Star Trek, or anything that implies I am a Jedi working for Sith, a desperate individual trying and failing to escape the Borg, or that the force has abandoned us to die. As apparently, one forwarded email that made the rounds was titled Attack of the Clones Server Error (as it was a literal clone server error), and non-geeks in management were very confused. Which is just the saddest thing I have ever heard.
However, on a lighter note, because embarrassment never gets old, there was an unexpected meeting in the room beside the break room the day I downloaded "Give Your Heart a Break". As that area is usually deserted, I had no qualms (quietly!) singing along (quietly, I swear) while doing something that might be considered a skip-hop-turn-slide-bounce past the break room, by the meeting, and down an empty cubicle row, perhaps several times. I was wearing headphones, which is my only defense for how I missed it, but apparently the end of the meeting was getting grumpy due to it going overtime and then the wobbling refrain drifted through the room by way of the half-open door and everyone came to the door to watch me.
Later that afternoon, amid a disturbing number of random smiles in my direction by people I did not know wandering around our area, I was informed of how much my performance was enjoyed. And my manager requested "Glad You Came" if I felt like doing a repeat.
Now I wonder every time I send an email if the first thing they think of when they see my name is the cubicle shuffle of pop music. And by that I mean, they totally do.
My life.
In case anyone is curious, I went for an eye exam earlier today and during the process there was a period of intense light per eye that has since developed into a headache that eased off just in time for me to reach terminal insomnia. This is fun for me.
tags: