December 1, 2021 at 5:20 pm
At my workplace, we actually use SSIS to pull data across from "live" systems into our reporting database. SOMETIMES it is a 1:1 copy of the data, sometimes we transform it for consumption by reports. The SSIS load is scheduled at different frequencies depending on report need. SOME SSIS packages run nightly, some run hourly, some run daily at 2:00 PM, others run every Friday. It depends on the report need and the duration of the loads. The nightly load runs during company downtime, so it can afford to be slow (takes hours sometimes). Hourly load runs during company uptime so needs to be fast. We have a 5 minute time limit on it. If it exceeds that, the DBA team gets notified and we look into why it is running slow. The 2 PM and Friday runs are always fast (under a minute), so we don't monitor those ones.
So it really depends on the need. If the report is run hundreds of times per minute, having the data refreshed "nightly" is probably a safe bet as hourly would cause interruptions to the reports based on how we do our SSIS load - truncate and reload each time rather than a merge approach. BUT it does depend on if end users need "live" data or snapshot data. For analytical reporting, snapshot data is usually what you want, but if your end users are expecting to see things they do appear immediately in the system, you will need a different approach than SSIS. In that case, some options I can think of are replication (I've not used this, but I think it should work), Service Broker (I've used this and it is a cool tool when it works, but can be challenging to debug when it acts up), 3rd party app that monitors for changes and pushes them across (in house or 3rd party... not sure of any that exist, but I am sure they do), and possibly other methods that I am not thinking of.
Yikes... 3 hours to fix what should be an easy problem is never fun. I understand having to work with different groups, but when a critical thing is busted, I like to set up a room with all people who can help and have laptops for everyone so they can work in the same room to check things out. Much faster to get things solved and with laptops for everyone, they can still RDP to their work boxes to do their job. Mind you I have only been involved in 1 major outage like that and I was not the coordinator in that scenario and we didn't have a big room - it was emails and phone calls and phone tag and it was slow and painful to find the problem and fix it and a lot of "We didn't change anything so it must be something you did"... finger pointing doesn't help, but in the end, the database side needed no changes so I didn't end up fixing anything... it was entirely the application team who released an update, told us they didn't, then later rolled back the update and the problem was fixed.
I feel bad you have to work with someone like that. There are times when I am sure I have unintentionally done things like that - I've merged stuff in git that blew out others work, but it was by accident and I worked with them to recover the work. People that go out of their way to make everyone else's job harder shouldn't have their job (my opinion). If you can't be a team player, then don't work on projects with others. And, again my opinion, a C# console app that is JUST for file/folder creation with specific names and such is definitely overkill. That feels like a job for powershell, or even a bat or cmd file.
The above is all just my opinion on what you should do.
As with all advice you find on a random internet forum - you shouldn't blindly follow it. Always test on a test server to see if there is negative side effects before making changes to live!
I recommend you NEVER run "random code" you found online on any system you care about UNLESS you understand and can verify the code OR you don't care if the code trashes your system.
December 2, 2021 at 2:34 pm
In this case it turned out to be the SQL Server's transaction log filled up the disk. That was finally discovered about 2 hours into the crisis. The DBAs handled it after that, then we were up and running an hour later.
What bothers me, a lot, is the walls that stand in the way. I used to advocate for lowering the walls but have given up that fight.
Kindest Regards, Rod Connect with me on LinkedIn.
December 2, 2021 at 6:04 pm
I agree... Our walls are low, but they still exist. And in some cases they should exist. For example, you wouldn't want me mucking around with AD or spinning up new VM's and I wouldn't want IT setting up new databases. THANKFULLY though, the IT team is easy to work with - I shoot them a message on teams and they will look into stuff for me. And if I can convince them I need access to something temporarily, they are willing to grant me access.
Another way our walls are "low" - I do database development, .NET coding, and DBA related work and the IT team trusts me enough to give me full admin on the server hosting SQL Server. So if the problem is related to an application I support, I can support it end-to-end. The downside is that I still follow the same process that a walled company would - reproduce the error on my machine (rule out end user or end user machine problems), next load up visual studio and find the chunk of code causing the problem and fix that (if possible). If not possible, jump into the database stored procedure and see what is happening. If it isn't due to the stored procedure (or I can't determine for certain if it is the stored procedure), start investigating the table data or I may need to jump over to the server and check disk space or firewall configuration or whatever... It is nice not having those "walls" as then I can solve most problems, but it also means more work for people like me LOL.
The above is all just my opinion on what you should do.
As with all advice you find on a random internet forum - you shouldn't blindly follow it. Always test on a test server to see if there is negative side effects before making changes to live!
I recommend you NEVER run "random code" you found online on any system you care about UNLESS you understand and can verify the code OR you don't care if the code trashes your system.
December 3, 2021 at 3:20 pm
At times it is really nice because you have the freedom to do the full development from end to end, but at other times it is painful because you start at the application and go "huh... this looks like a DBA problem... I'll go bug the DBA" and you get to switch hats to DBA mode and go "nope, this is definitely in the application side. Database is working fine."
Joking aside, I like working here. There are a TON of learning opportunities and I am picking up new tricks all the time. Only downside is that you never REALLY become an expert in any of the things you work with because you need to know how to support so much stuff. And as soon as I get close to feeling like I have mastered a tool, a new tool falls in my lap OR someone asks for something that is outside my knowledge on the tool. On top of the application development and database work, I also administer the Jira and Confluence systems and do some work with Salesforce as well. Salesforce work is pretty light though - mostly just the "repeated tasks" that are easy to follow a script to get them done and new user stuff, which is also not too challenging.
Recently did I stop doing windows updates and got those pushed off to the IT department who can automate it. I had been doing it manually as there were odd cases where windows updates would break a SQL instance, but we have good monitoring set up so there is no reason we can't automate it and I get an alert when the system reboots and things failover and an alert when things fail back to the primary. So no need for me to manually reboot the servers (physical servers) and then panic a little as it takes 20 minutes (on average) per reboot due to windows updates.
I work for Vecima Networks in Canada, but we do have offices all over the place and we are hiring. Unfortunately, not for my team, but we do have job openings!
The above is all just my opinion on what you should do.
As with all advice you find on a random internet forum - you shouldn't blindly follow it. Always test on a test server to see if there is negative side effects before making changes to live!
I recommend you NEVER run "random code" you found online on any system you care about UNLESS you understand and can verify the code OR you don't care if the code trashes your system.
Viewing 5 posts - 16 through 19 (of 19 total)
You must be logged in to reply to this topic. Login to reply