Tag Archive for: commands

I guess it all comes to our experiences so far. I still find it impossible to automate user tasks on a server. Maybe it has to do with the diversity of the applications that they are already installed. Keep in mind that if an administrator takes that road he must also test/reconfigure/update the automation processes that he already created. Yes some simple login tests can be automated but still you add up extra workload....multiply that by 1000+ ...

Concerning the pre-prod environment (when one is present) , the key users also dont want to spend extra time testing the updates on your test environment. For them it is your test environment, not theirs. "Their" pre-prod is valuable even more than the live one....and you should treat it like the production one as well :-/ I would also like to mention that sometimes the pre-prod environment does not exist in the corporate Network.More than 50% of the applications that run on our Servers come directly from external partners that just install them and update them without even sometimes letting you know.Imagine how difficult it is to keep track of the documentation, changes and the automated users tasks you mentioned before.

With the backups themselves it depends on the administrator and how far is he willing to go.My approach to that is the following(considering that i am the sole responsible if something goes wrong):

Every backup system has a database.All that an administrator needs is a read-only account on that database.Then with a simple SQL query you can see if there is a succesfull backup for your servers. Simple and fast (i am already doing that with Sesam, Veeam and Netbackup). Another solution would be to ask for a daily report.It is not that big deal to receive the backup report as well. Still doing the SQL thing though because i automate it with PS.

thank you for your reply. It is always nice to have a solid conversation.

BUT having a test environment similar to your production one is simply impossible.

Can't agree on this one. Its very easy to have identical servers/applications. What differs is the load. You can have automated tests which simulate standard user behavior. With LOB applications its common to have key users that do the tests in the test/pre-prod environment. Thats the only way how an functional update can be validated.
When we patched business critical application (not only from OS point of view, but also the application itself) we always had a Key user who did the app test and a Local IT support guy who did OS related tests.
Yes, system admins do not always know the app itself, thats why I said its responsibility of system AND app owners 🙂

With the backups its not that easy. Its the same like the system/app owner relation... In large organizations, Windows admins have no access to backup systems (Tivoli, etc etc). You cant ask backup team if your 3000 servers that you are going to patch were backed up successfully.
Cheers

Very usefull piece of Information. Thank you.
Since it is mentioned in various articles , guides etc i have to comment on the existence of a test environment. Yes having a test environment is a must and it should be used exactly like mentioned above , BUT having a test environment similar to your production one is simply impossible. A server in the test environment will never be even close to a "live" one. Even if you clone the server and make the copie available in the test environment , the administrator cannot make sure that the server is used like the live one or even test the server himself.
We are unaware of the specific details of the applications that run on a server and only a real user or an App Admin can really tell the impact (if there was) that an update had. Real users don't connect to our test environment. They connect to the production one.
What are we testing then? Mostly Server stability (BSOD, reboots etc) or known issues. What kind of precautions do we take? We take snapshots or make sure that a snapshot exists. We also make sure that we take backups (or check if they exist) of crucial Applications (such as databases ) that are not covered from a snapshot.
I also believe that no matter how big is an organisation, the administrator who performs the patch management should always be aware of the backup procedures and be always in contact with the backup team.