June 28, 2021 at 9:17 pm
Sorry to pick this one apart Luis. I know you were posting here hoping to find some leads, not a critique of the posting that you didn't write. Hopefully you are able to find somebody soon.
Don't worry about that. I should probably give this feedback to my manager (without mentioning names) as it makes sense that people that actually know could be scared away.
June 29, 2021 at 2:16 am
Do you have an actual example of a 30 million row transaction where every row was actually required or you had to fail all 30 million rows? I'm not looking for something hypothetical... I'm looking for an actual example that you've personally experienced.
I cannot remember exact number of records in those files, it was most certainly less than 30 million, but definitely exceeding 3 million "uninvoiced deliveries" to certain customers per one XML file, which we've been loading into what used to be "AirBP eNabler".
It took more than 3 hours for BizTalk to just parse those files and dump the content into temporary tables for further processing.
And yes, those XML files had, apart from "Header", a section named "Summary", which contained all sorts of aggregations of the data in the file: total deliveries, total items delivered, total volume, etc.
The ETL system marked all those 3+million deliveries as "successfully submitted" upon the file uploading.
Of course, if any of the records failed to upload for any reason, we could not change the numbers in the file's summary. If we'd partially accepted the file the ETL system still would have "success" status assigned to the number of entries mentioned in the Summary.
We had to reject the whole file, and in the response file we did our best identifying the issue as detailed as possible, so the ETL guys could either correct or exclude the faulty entry and submit the better version of that file.
_____________
Code for TallyGenerator
June 29, 2021 at 4:53 pm
Jeff Moden wrote:Do you have an actual example of a 30 million row transaction where every row was actually required or you had to fail all 30 million rows? I'm not looking for something hypothetical... I'm looking for an actual example that you've personally experienced.
I cannot remember exact number of records in those files, it was most certainly less than 30 million, but definitely exceeding 3 million "uninvoiced deliveries" to certain customers per one XML file, which we've been loading into what used to be "AirBP eNabler".
It took more than 3 hours for BizTalk to just parse those files and dump the content into temporary tables for further processing.
And yes, those XML files had, apart from "Header", a section named "Summary", which contained all sorts of aggregations of the data in the file: total deliveries, total items delivered, total volume, etc.
The ETL system marked all those 3+million deliveries as "successfully submitted" upon the file uploading.
Of course, if any of the records failed to upload for any reason, we could not change the numbers in the file's summary. If we'd partially accepted the file the ETL system still would have "success" status assigned to the number of entries mentioned in the Summary.
We had to reject the whole file, and in the response file we did our best identifying the issue as detailed as possible, so the ETL guys could either correct or exclude the faulty entry and submit the better version of that file.
The bottom line is as it always is... "It Depends" on what is needed. 😀
--Jeff Moden
Change is inevitable... Change for the better is not.
June 30, 2021 at 6:44 am
A while ago I was looking for a new job. A friend of mine connected me with a company that he worked at their DBA team at the past. After few interviews, I was hired. About a year after I started working there, we needed to hire another DBA. My manager posted an ad and I looked at it. That ad had nothing to do with what we needed. We did not have a single DBA in our team that met half the requirements in the ad. I asked my boss why did he post such an ad and he told me, that this is the ad that they always post when they are looking for a DBA and it is the same ad that they posted when I was interviewed. I told him that I never saw the ad, because the connection between us was done by another DBA, but if I would have seen that ad, most chances are that I would have not applied to this job because according to the ad, I'm not qualified for that job. Never did understand ads that have nothing to do with reality.
June 30, 2021 at 2:21 pm
The bottom line is as it always is... "It Depend" on what is needed. 😀
Yep, too right.
It depends, and it depends on requirements, not on file size or any other property of the data channel.
_____________
Code for TallyGenerator
June 30, 2021 at 2:27 pm
Never did understand ads that have nothing to do with reality.
How many ads did you actually see?
How many of them passed reality check?
Any?
_____________
Code for TallyGenerator
June 30, 2021 at 3:26 pm
Ads always ask for more than the job. I don't know that I've seen one that was accurate since I stopped tending bar.
If you think you have to meet every item before you apply, you're letting opportunities go. This isn't the fault of the ad. It's the way you're looking at the ad. Same for many others.
June 30, 2021 at 5:23 pm
Ads always ask for more than the job. I don't know that I've seen one that was accurate since I stopped tending bar.
If you think you have to meet every item before you apply, you're letting opportunities go. This isn't the fault of the ad. It's the way you're looking at the ad. Same for many others.
Actually, it IS the fault of the ad. Just look at the damned things. People need to ask for what they actually need instead of the pie in the sky plus the kitchen sink BS they typically ask for.
--Jeff Moden
Change is inevitable... Change for the better is not.
June 30, 2021 at 7:48 pm
Ads always ask for more than the job. I don't know that I've seen one that was accurate since I stopped tending bar.
If you think you have to meet every item before you apply, you're letting opportunities go. This isn't the fault of the ad. It's the way you're looking at the ad. Same for many others.
I'm reminded of when I applied / was headhunted for my current job. The list of what they WANTED would've been enough to scare me off if I'd come across the listing, I opted to give it a go though.
Worked out pretty well for me, even if I did have to crash course via Google how to at least keep an Oracle Database running for a couple months.
But yeah, I've seen plenty of comments about job listings that sound like they're looking for a "one-stop-shop" for everything including building maintenance...
June 30, 2021 at 8:28 pm
I "love" some of the ads I've seen... they start with needing only 2 years of experience but want a full stack developer that is also an expert at database design, keeping the entire application and multiple servers performance tuned, must be an expert in 3 different versions of "C", anything and everything having to do with Java ,Python, PowerShell, and more, must be an expert in SQL Server, Oracle, Postgre, MySql, Access, and AWS Redshift, must be an expert in Power BI, SSIS, SSRS, SSAS, SSSB, and a couple of other 4 letter words, must be an expert mentor in everything having to do with all the SQL for all the previously mentioned RDBMSs, must be an expert in troubleshooting and designing all code. must be an expert in things like partitioning, indexes, index maintenance, an expert at anything and everything having to do with the SAN and S3 storage, have a good understanding of Azure as both a DBA and and a Developer, must be able write all testing, and write the company standards at the same time with gathering requirements from customers, must have a minimum of a Bachelor's degree in computer science AND statistics, but a Master is preferred, and must have multiple certifications including Cisco certifications and be a Windows ninja, must be able to do updates to all of that, must take part in 24/7 on-call although you'll be the only one working there, and it's an entry level job with salary commensurate to experience with a cap at 60K.
Yeah... let's apply for that. I wouldn't apply for that for 300K per year. In fact, I actually turned one down for 325K. It was just stupid and anyone that writes such an ad is stupid. Period.
--Jeff Moden
Change is inevitable... Change for the better is not.
June 30, 2021 at 8:53 pm
I "love" some of the ads I've seen... they start with needing only 2 years of experience but want a full stack developer that is also an expert at database design, keeping the entire application and multiple servers performance tuned, must be an expert in 3 different versions of "C", anything and everything having to do with Java ,Python, PowerShell, and more, must be an expert in SQL Server, Oracle, Postgre, MySql, Access, and AWS Redshift, must be an expert in Power BI, SSIS, SSRS, SSAS, SSSB, and a couple of other 4 letter words, must be an expert mentor in everything having to do with all the SQL for all the previously mentioned RDBMSs, must be an expert in troubleshooting and designing all code. must be an expert in things like partitioning, indexes, index maintenance, an expert at anything and everything having to do with the SAN and S3 storage, have a good understanding of Azure as both a DBA and and a Developer, must be able write all testing, and write the company standards at the same time with gathering requirements from customers, must have a minimum of a Bachelor's degree in computer science AND statistics, but a Master is preferred, and must have multiple certifications including Cisco certifications and be a Windows ninja, must be able to do updates to all of that, must take part in 24/7 on-call although you'll be the only one working there, and it's an entry level job with salary commensurate to experience with a cap at 60K.
Yeah... let's apply for that. I wouldn't apply for that for 300K per year. In fact, I actually turned one down for 325K. It was just stupid and anyone that writes such an ad is stupid. Period.
And this on top of the required 10+ years of experience in technologies only introduced in the last two years
😎
Obviously missing from your list Jeff, are the pilot qualifications, at least a LearJet, Citation and a couple of others that aren't in production yet.
June 30, 2021 at 9:23 pm
Reading through some of these comments jogged the old memory bank. When I was applying for my current job my company had two separate adds. The jobs sounded similar but each had a a few requirements in common but from my point it looked like two separate jobs. When I was interviewing I asked about the other listing. They told me it was for the same position I was applying for, they had just wanted to cast a wider net. Didn't want to scare away some from applying. I guess that's one way to do it.
-------------------------------------------------------------
we travel not to escape life but for life not to escape us
Don't fear failure, fear regret.
July 1, 2021 at 3:37 am
Jeff Moden wrote:I "love" some of the ads I've seen... they start with needing only 2 years of experience but want a full stack developer that is also an expert at database design, keeping the entire application and multiple servers performance tuned, must be an expert in 3 different versions of "C", anything and everything having to do with Java ,Python, PowerShell, and more, must be an expert in SQL Server, Oracle, Postgre, MySql, Access, and AWS Redshift, must be an expert in Power BI, SSIS, SSRS, SSAS, SSSB, and a couple of other 4 letter words, must be an expert mentor in everything having to do with all the SQL for all the previously mentioned RDBMSs, must be an expert in troubleshooting and designing all code. must be an expert in things like partitioning, indexes, index maintenance, an expert at anything and everything having to do with the SAN and S3 storage, have a good understanding of Azure as both a DBA and and a Developer, must be able write all testing, and write the company standards at the same time with gathering requirements from customers, must have a minimum of a Bachelor's degree in computer science AND statistics, but a Master is preferred, and must have multiple certifications including Cisco certifications and be a Windows ninja, must be able to do updates to all of that, must take part in 24/7 on-call although you'll be the only one working there, and it's an entry level job with salary commensurate to experience with a cap at 60K.
Yeah... let's apply for that. I wouldn't apply for that for 300K per year. In fact, I actually turned one down for 325K. It was just stupid and anyone that writes such an ad is stupid. Period.
And this on top of the required 10+ years of experience in technologies only introduced in the last two years 😎
Obviously missing from your list Jeff, are the pilot qualifications, at least a LearJet, Citation and a couple of others that aren't in production yet.
And the ever popular "turn water to wine" requirement (although some of my favorite Developers and fellow DBAs were really good about turning water into some excellent beer).
--Jeff Moden
Change is inevitable... Change for the better is not.
July 1, 2021 at 4:17 am
Shifting gears to another aspect to employment, I'd say "and so it begins" except that it's been this way for a long time. It's just getting worse. If you have one of those "self evaluation/set your own goals" thingies going on at work, good luck. The one in the article below doesn't even allow that.
--Jeff Moden
Change is inevitable... Change for the better is not.
July 1, 2021 at 9:40 am
And the ever popular "turn water to wine" requirement (although some of my favorite Developers and fellow DBAs were really good about turning water into some excellent beer).
I wish it would be "turn water into wine".
It's rather "perform specifically shaped manipulations over vessels with certain liquids, reading these and these prays in that particular order". And it does not matter that those manipulations did not turn water into wine before, but they've read in some white papers they should, so they just look for a replacement for the previous guy, who obviously was doing it wrong, hoping the new one can finally do it properly.
I was once hired for 6 month contract to develop and deploy series of DTS tasks and (in the side) tune up several problematic queries in their database. In the following 8 years in the company I deployed 3 DTS packages, 2 of which I happily decommissioned myself couple of years later. Apparently, DTS packages was what they wanted, but not what they needed.
Enormous list of scripting tools and frameworks only indicates the mess in the company's IT structure and the CIO head, lack of leadership and architectural approach. They jump from tool to tool in a search of a silver bullet in a search of a silver bullet, which, of course, nowhere to be found. Because, instead of declaring the goals to be achieved and trust the professionals to know how to it better, they produce instructions for coding monkeys what and when to use.
I was myself turned away from a company because the hiring manager wanted somebody who would follow his instruction, not someone who has a professional opinion and can decide for himself how to achieve the goals. Probably it was a good thing. Not so good for those who rely on data analysis from that company.
_____________
Code for TallyGenerator
Viewing 15 posts - 65,536 through 65,550 (of 66,547 total)
You must be logged in to reply to this topic. Login to reply