SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


The Best Database Administrators Automate Everything


The Best Database Administrators Automate Everything

Author
Message
ryan.mcatee
ryan.mcatee
Valued Member
Valued Member (71 reputation)Valued Member (71 reputation)Valued Member (71 reputation)Valued Member (71 reputation)Valued Member (71 reputation)Valued Member (71 reputation)Valued Member (71 reputation)Valued Member (71 reputation)

Group: General Forum Members
Points: 71 Visits: 134
John.Sansom (4/9/2012)
Hi Ryan,

Consider that the article theme is to encourage the development of a mind set that looks to automate all processes, where applicable. It is not the aim of the article to explicitly identify what those processes are, as these are too varied from one Data Professional role to another i.e. the processes are often context specific.

How would you suggest that the article could provide what you would consider to be sufficient depth, whilst at the same time not being process/scenario specific?

Thanks for your comments.



I understand. The article left me wondering "are you talking about me, or just system admins that do database backups and whatnot?" After re-reading the article (slowly this time), it appears to be written for DBAs, but I see now these concepts can apply to everyone in the field.
John.Sansom
John.Sansom
Right there with Babe
Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)

Group: General Forum Members
Points: 799 Visits: 1558
Thanks for your feedback Ryan! If something is not clear then I want to know about it and I appreciate you taking the time to explain your thinking.


John Sansom (@sqlBrit) | www.johnsansom.com
Pedro Palhoto
Pedro Palhoto
Forum Newbie
Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)

Group: General Forum Members
Points: 3 Visits: 9
John.Sansom (4/6/2012)
Wait, do people even still use Python? ;-)


I have recently come back to DBAing and I was about to use Python to automate most of the tasks. From this thread's discussion I'd better take a better look into Powershell. Currently we don't have access to SSIS (long story) and that is why a full blow language/environment may come in handy. Anyway, here is a list of Python tools/packages I gathered for the current required reporting tasks I have to attend. This might come in handy for someone.

MS-SQL Server connectivity:
http://code.google.com/p/pymssql/
http://code.google.com/p/pyodbc/

Data Warehouse Extract-Transform-Load (ETL):
http://pygrametl.org/

Object-Relational Mapping Frameworks:
http://elixir.ematia.de/trac/wiki
http://www.sqlalchemy.org/
http://sqlobject.org/

Python Language Integrated Query – Expression Trees:
https://github.com/heynemann/pynq/wiki (Pynq)
http://www.codebadger.com/blog/post/2009/06/01/Pythone28099s-LINQ-Equivalents-e28093-filter%28%29-map%28%29-and-list-comprehension.aspx (Native Python LINQ equivalents)

Reporting in Python:
http://www.geraldoreports.org/docs/index.html
https://github.com/lightcaster/xlrep

Python integration with other languages/libraries/architectures:
http://jpype.sourceforge.net/ (Java)
http://pythonnet.sourceforge.net/readme.html (.NET)
http://sourceforge.net/projects/pywin32/?_test=b (Win32 Extensions)
http://www.scipy.org/Weave (C/C++ inline)

Excel and CSV File Integration:
http://stackoverflow.com/questions/4257771/python-writing-to-excel-2007-files-xlsx-files
http://packages.python.org/Pyvot/tutorial.html
http://docs.python.org/library/csv.html

Portable Apps (if you need them):
http://www.portablepython.com/
http://pydev.org/ (+ Eclipse Portable http://portableapps.com/node/28647 )
John.Sansom
John.Sansom
Right there with Babe
Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)

Group: General Forum Members
Points: 799 Visits: 1558
Thanks for sharing those resources Pedro! I'm sure they will prove useful to other readers.


John Sansom (@sqlBrit) | www.johnsansom.com
pkrudysz
pkrudysz
SSC Journeyman
SSC Journeyman (80 reputation)SSC Journeyman (80 reputation)SSC Journeyman (80 reputation)SSC Journeyman (80 reputation)SSC Journeyman (80 reputation)SSC Journeyman (80 reputation)SSC Journeyman (80 reputation)SSC Journeyman (80 reputation)

Group: General Forum Members
Points: 80 Visits: 267
Good article, but rather obvious (who would have thought - automate :-) )

One sentence that has value to me is: "The key thing with automation contrary to typical DBA practices is to not necessarily use the most appropriate tool or technology for the job but to use whatever tool that you can to get the job automated and done.". That is a good point.
John.Sansom
John.Sansom
Right there with Babe
Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)

Group: General Forum Members
Points: 799 Visits: 1558
Indeed obvious and yet so tricky for many to actually do....

Thanks for your comments!


John Sansom (@sqlBrit) | www.johnsansom.com
bobpak
bobpak
SSC Rookie
SSC Rookie (48 reputation)SSC Rookie (48 reputation)SSC Rookie (48 reputation)SSC Rookie (48 reputation)SSC Rookie (48 reputation)SSC Rookie (48 reputation)SSC Rookie (48 reputation)SSC Rookie (48 reputation)

Group: General Forum Members
Points: 48 Visits: 626
Best careful what you ask for. You cannot automate every single thing and you might lose your job as a result.



tomex.iskandar
tomex.iskandar
Forum Newbie
Forum Newbie (4 reputation)Forum Newbie (4 reputation)Forum Newbie (4 reputation)Forum Newbie (4 reputation)Forum Newbie (4 reputation)Forum Newbie (4 reputation)Forum Newbie (4 reputation)Forum Newbie (4 reputation)

Group: General Forum Members
Points: 4 Visits: 37
I feel relaxed and excited my applications has successfully passed a 3 years working tool. It is a fully automated ETL, Data mining and Data warehouse system. It consumes a multiple daily data sources (zipped files at FTP and Json at a RESTful webserver) and process and store for more than 120 columns for the business requirement. All processes are automatic! It is a 30,000 lines VBA codes, 10,000 line TSQL code and working MySQL Server. ADO is the most technology used in the applications with a few uses of bat scripting (ohh... I learnt this from 15 years ago), vbscripting, lotus notes class etc. It is beyond SSIS and SSAS capabilities. A wise man (BS) says 'There are no “cookbook” methods that can replace intelligence, experience, and good taste in design and programming'. Aha that man is very true and I did it for fun.

Well I told my kid and my mum that I have made a robot. Yes this robot is very productive and diligent compared to human being, but sometime it is clever than me as well. I hate a salesman who keeps preaching me that SSIS and SSAS is the best way and every day he pushes/or executes buttons so many time to get things done and never realizes he is a robot. The good thing is this robot has stopped him preaching me out.
John.Sansom
John.Sansom
Right there with Babe
Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)Right there with Babe (799 reputation)

Group: General Forum Members
Points: 799 Visits: 1558
bobpak (8/21/2014)
Best careful what you ask for. You cannot automate every single thing and you might lose your job as a result.


You're more likely to get promoted into a new role, poached by a more ambitious company or have moved on of your own accord, as you continually look for new ways to stretch yourself and grow.


John Sansom (@sqlBrit) | www.johnsansom.com
cjsommer
cjsommer
SSC Rookie
SSC Rookie (26 reputation)SSC Rookie (26 reputation)SSC Rookie (26 reputation)SSC Rookie (26 reputation)SSC Rookie (26 reputation)SSC Rookie (26 reputation)SSC Rookie (26 reputation)SSC Rookie (26 reputation)

Group: General Forum Members
Points: 26 Visits: 502
bobpak (8/21/2014)
Best careful what you ask for. You cannot automate every single thing and you might lose your job as a result.


Your statement is the reason a lot of companies carry so much technical debt. People think that if they build themselves into a process they can't be replaced, which is completely false. I've been in big companies and I've been in small companies, and no matter how much I have automated, I have never run out of things to do.

Think of it from a management standpoint. Who would you rather have on your team? The DBA who takes 3 hours to manually refresh a DEV database, or a DBA who automates it so developers can restore their own DEV system? I know my choice.

Your value to your company is finding innovative ways to better serve your customers.


John Sansom, I thought this article looked familiar. I agree 100%. Glad they re-posted it.
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search