May 16, 2023 at 4:08 pm
Jonathan AC Roberts wrote:Nearly billion claims every day seems a very high number for Medicaid Florida, considering the population of Florida is only about 22 million.
That sounds about right. The number of medicaid recipients in Florida is very high. Without looking, I think they have the highest percentage of medicaid recipients per capita of all the states.
A regular trip to the doctors office results in multiple claims.
Side note on Florida medicaid. In 1999, they did an audit of our billings. They determined that we had underbilled them 700k over three years. So the company got fined 300k. The issue was the module that generated the bills. The programmer who wrote the code did not consider the time portion of the datetime columns, so what ever date was the last date of the billing report was never included.
But that's nearly 50 claims a day for each person in Florida.
May 16, 2023 at 4:38 pm
Michael L John wrote:Jonathan AC Roberts wrote:Nearly billion claims every day seems a very high number for Medicaid Florida, considering the population of Florida is only about 22 million.
That sounds about right. The number of medicaid recipients in Florida is very high. Without looking, I think they have the highest percentage of medicaid recipients per capita of all the states.
A regular trip to the doctors office results in multiple claims.
Side note on Florida medicaid. In 1999, they did an audit of our billings. They determined that we had underbilled them 700k over three years. So the company got fined 300k. The issue was the module that generated the bills. The programmer who wrote the code did not consider the time portion of the datetime columns, so what ever date was the last date of the billing report was never included.
But that's nearly 50 claims a day for each person in Florida.
Michael L John wrote:Jonathan AC Roberts wrote:Nearly billion claims every day seems a very high number for Medicaid Florida, considering the population of Florida is only about 22 million.
That sounds about right. The number of medicaid recipients in Florida is very high. Without looking, I think they have the highest percentage of medicaid recipients per capita of all the states.
A regular trip to the doctors office results in multiple claims.
Side note on Florida medicaid. In 1999, they did an audit of our billings. They determined that we had underbilled them 700k over three years. So the company got fined 300k. The issue was the module that generated the bills. The programmer who wrote the code did not consider the time portion of the datetime columns, so what ever date was the last date of the billing report was never included.
But that's nearly 50 claims a day for each person in Florida.
Yeah. After I posted this I did the math. It does not work. Now, each claim is made up of multiple details. That may be where the billion is coming from.
Michael L John
If you assassinate a DBA, would you pull a trigger?
To properly post on a forum:
http://www.sqlservercentral.com/articles/61537/
May 16, 2023 at 4:49 pm
Yes Jonathn AC Roberts and Python will let you do multi-threading as well but have you ever looked into how how MS Windows supports their so-called multi-threading, especially since windows does not support Forking? So what is multi-threading in MS Windows -- it is a queue of requests operated on in a linear fashion -- which by my understanding of multi-threading that it is not. As you do not have multiple threads but one single thread with multiple processing being run as it can get to it. If you have one thread doing a continuous stream you have just blocked or negated your ability to do their so-called multi-threading within MS Windows. How do I know this, by having encountered it and then having eventually figured out how to implement multi-processing.
Yes I now that nearly a billion claims seems rather high to me but that was the number I understood we were processing everyday. And as Michael L John points out a single office visit creates multiple claims and a single hospital visit creates a mass of claims. So yeah about 50 claims per person (taking the full current population of Florida) still to me seems a bit high but the numbers were the numbers that is all I can say.
Now I cannot say if that number might not have been the number of individual claims but maybe the number of processes, or included the number of times an individual claim had to be processed. As we did have several stages to processing a single claim which had to be done in a sort of sequence and each claim or batch of claims was not handled exactly the same within each sequence. Still I know some of the processing took a looong time to complete due to the shere volume of claims that were being processed on a daily basis. Further certain end reports I think actually took more than a day to produce due to this shere volume as well. That being said I am working with the VA now on their claims processing and while not nearly the volume of Florida's Medicaid it is still a rather daunting number of claims that need to be processed and they like us so much they have been seriously talking about having us process all of the VA claims for the country which we figure would mean about 5 times as many as we are processing now and that means an issue if we cannot get some of our latency issues under control. Something I have been working on since I got here.
May 16, 2023 at 8:03 pm
Some of the claims are still coming from this problem I bet:
https://www.cbsnews.com/news/medicare-fraud-a-60-billion-crime-23-10-2009/
May 17, 2023 at 10:29 pm
Whether you know SQL or not, do you have access to a database? There are plenty of devs that don't really have access to anything other than LocalDB (that they control), and so they might not think of SQL as the place to clean data, not because it can't, but because creating a database, loading data, etc. is hard. At least to them.
With a little R or Python, you can quickly load and analyze data. especially if you are just doing analysis quickly and not saving things. I would argue that you don't need to know it, but it can be helpful and easier.
As far as loading billions of rows, what's the memory size needed? Might be a lot, might not be. Python could handle this for sure, if you have enough RAM on a machine. It's also much easier than C/C++ because y0u don't need to compile it. Is it faster? probably not, though the way lots of people write C++ programs, any execution savings would be lost in time to write/debug code.
May 18, 2023 at 1:54 am
Well Steve Jones perhaps we will have to agree to disagree on that. I am not just a SQL developer but I have used numerous languages over the years including C, C++, C#, Java, Python, and a lot more. Further I did not find Python to be any more difficult or any easier than the various other languages it simply was just another language, and the language that those I was working for were using. Further currently those that I am working with use Java and SQL and we try to offload the data processing to the Java side of things (compiled C-based language) and the data collection and storage to the SQL side of things. Further, this is a common approach for large databases as working with that data needs to be done quickly. However, if you are working with a much smaller dataset then T-SQL will work just fine for just about all of it. I draw this conclusion from working on many different sizes of native database languages for instance VBA in conjunction with MS Excel perhaps the smallest database out there. VBA with MS Access. Python with MySQL which does not support stored procedures. As well as numerous other combinations. The only time I would use Python or any language for that matter with a database, is if that database does not support the various aspects that MS Sequel Server or Oracle have incorporated into their databases or if I am working with large scale data and need a much faster processing language at which point I would not select Python as that would be a step backwards rather than forwards.
All I am trying to say with all of that, is be careful what you recommend to others as they might go down the wrong rabbit hole based on that advise and suggesting that someone use Python or R instead of the Sequel Server natural environment is definitely advising someone to go down the wrong rabbit hole.
By the way has anyone else noticed that this discussion has gone way off topic??
May 18, 2023 at 2:37 am
By the way has anyone else noticed that this discussion has gone way off topic??
It became a "water cooler" thread very early in its existence. 😉 Most of us old timers simple refer to it as "The Thread". 😀
--Jeff Moden
Change is inevitable... Change for the better is not.
May 18, 2023 at 3:23 am
I agree with Eirikur on all this... I believe that a whole lot of people resort to the likes of Python because they don't actually know how to use SQL Server/T-SQL.
Another reason is they believe in the total BS out there like "To a hammer, everything is a nail" without understanding that when you're trying to drive nails, you should use some sort of hammer. The other extremely tired saying of "Just because you can do something in SQL, doesn't mean you should". Pure Rubbish. It should be "Just because you can't even spell SQL, doesn't mean you shouldn't learn how". It is, after all, the language of structured data. Perhaps we should start the new mantra of "Just because you can do something in Python (Powershell, C#, VBA, flavor-of-the-day, etc) , doesn't mean you should".
--Jeff Moden
Change is inevitable... Change for the better is not.
May 18, 2023 at 3:27 am
Just a post so that the 1st post on this page actually shows up. I wish they'd fix that and also finally get the pages things fixed, which is simply a problem of posts being marked as SPAM not being counted for the page counts.
--Jeff Moden
Change is inevitable... Change for the better is not.
May 18, 2023 at 1:51 pm
... Perhaps we should start the new mantra of "Just because you can do something in Python (Powershell, C#, VBA, flavor-of-the-day, etc) , doesn't mean you should".
Love it, 100% agree, two thumbs up, you hit the nail on the head with that statement(Ha Ha). Exactly how I feel Jeff. 🙂
-------------------------------------------------------------
we travel not to escape life but for life not to escape us
Don't fear failure, fear regret.
May 18, 2023 at 3:51 pm
...
All I am trying to say with all of that, is be careful what you recommend to others as they might go down the wrong rabbit hole based on that advise and suggesting that someone use Python or R instead of the Sequel Server natural environment is definitely advising someone to go down the wrong rabbit hole.
I wouldn't advise anyone to use or not use Python. You can disgree with my points, but they're from your perspective and the problem space in which you're working. If you have an application and you are connecting to it to do data analysis across large volumes, then likely SQL works well. If you've set some things up, this is great.
If you do ad hoc analysis, or you have people that are semi-technical and want to often change the way they might look at data, especially with some visualizations, then I might not think that. If someone sends me a CSV of data from somewhere, I think of these things:
If I wanted to do something and show people, I might pick R or Python if I did this type of thing, not with this file, but this type of thing, often. I can quickly load into memory, I can manipulate data, I can create visuals, and I can drop my code in a notebook that others can use to do the same thing or see results/visuals.
In an ad hoc, lightweight way, I think R or Python are much better. In fact, I had to load the Summit sessions from a CSV at one point and I tried both SQL and R. R was much cleaner and quicker, even with my having to look up a bunch of syntax.
It's not one or the other, but asking people what they are trying to do before deciding on how. I assume that's what most of you do here. You don't just say "use SQL Server". Even Jeff, who beats on other tech, thinks about why. He has reasons why in many cases, using C# or Python or PoSh, isn't better than SQL.
May 18, 2023 at 3:55 pm
Just a post so that the 1st post on this page actually shows up. I wish they'd fix that and also finally get the pages things fixed, which is simply a problem of posts being marked as SPAM not being counted for the page counts.
me too. I have complained and haven't gotten anyone to figure this out. I also feel allergic to PHP and haven't wanted to try and decode that.
May 18, 2023 at 3:55 pm
Jeff Moden wrote:... Perhaps we should start the new mantra of "Just because you can do something in Python (Powershell, C#, VBA, flavor-of-the-day, etc) , doesn't mean you should".
Love it, 100% agree, two thumbs up, you hit the nail on the head with that statement(Ha Ha). Exactly how I feel Jeff. 🙂
The caveat to this is that if you don't know how to do it in SQL but do know Python/R/etc., then use that. However, if it doesn't scale, learn SQL.
May 18, 2023 at 4:20 pm
Dennis Jensen wrote:... All I am trying to say with all of that, is be careful what you recommend to others as they might go down the wrong rabbit hole based on that advise and suggesting that someone use Python or R instead of the Sequel Server natural environment is definitely advising someone to go down the wrong rabbit hole.
I wouldn't advise anyone to use or not use Python. You can disgree with my points, but they're from your perspective and the problem space in which you're working. If you have an application and you are connecting to it to do data analysis across large volumes, then likely SQL works well. If you've set some things up, this is great.
If you do ad hoc analysis, or you have people that are semi-technical and want to often change the way they might look at data, especially with some visualizations, then I might not think that. If someone sends me a CSV of data from somewhere, I think of these things:
- I need a SQL instance and database
- I have to use some clunky import method, most of which are somewhat poor in SQL Server tooling
- I then need to run a bunch of SQL statements to clean or reformat data
- I can run queries, but visuals are few
- I can use Power BI, but then that's another whole project beyond simple analysis
If I wanted to do something and show people, I might pick R or Python if I did this type of thing, not with this file, but this type of thing, often. I can quickly load into memory, I can manipulate data, I can create visuals, and I can drop my code in a notebook that others can use to do the same thing or see results/visuals.
In an ad hoc, lightweight way, I think R or Python are much better. In fact, I had to load the Summit sessions from a CSV at one point and I tried both SQL and R. R was much cleaner and quicker, even with my having to look up a bunch of syntax.
It's not one or the other, but asking people what they are trying to do before deciding on how. I assume that's what most of you do here. You don't just say "use SQL Server". Even Jeff, who beats on other tech, thinks about why. He has reasons why in many cases, using C# or Python or PoSh, isn't better than SQL.
To be sure, my point was directed at people that tout and readily broadcast in a seriously passive aggressive manner that "Just because you can to something in SQL, doesn't mean you should" as a defense for some of the things that they use other technologies for even though they're the ones that are missing a point.
As you say, I do consider the Brent Ozar mantra of asking "What problem are you trying to solve?", especially if I'm the one trying to solve a problem.
An extreme example of this is that I have, in the past (long time ago and the code is long gone, thank goodness), used T-SQL to do a little screen scraping for data. It was a huge PITA and not efficient and I would never recommend that as an alternative to using Python for such a thing. At least I knew. In high contrast, a good number of rabid Python supporters that I've recently run into regularly bash SQL for things that SQL is actually much better at. The problem is, they're doing the bashing without actually knowing. They're just repeating the same crap that other supposed "experts" have also incorrectly stated. The other problem is, some of them are doing such in very public places (like the question I recently posted on this thread about a statement someone made) and that's going to be absorbed by people new to the world of data. In other words, it perpetuates the very wrong idea that "SQL is just a place to store some data".
Of course, that same kind of "expert" testimony happens a lot in the world of SQL. One of my recent favorites is people demonstrating how to combine a DATE and TIME column using a conversion to DATETIME for both and adding them together. Not only is there a loss of resolution but there's some pretty nasty rounding that can "move" the data to the next hour, day, week, month, and even year.
--Jeff Moden
Change is inevitable... Change for the better is not.
May 18, 2023 at 4:24 pm
Jeff Moden wrote:Just a post so that the 1st post on this page actually shows up. I wish they'd fix that and also finally get the pages things fixed, which is simply a problem of posts being marked as SPAM not being counted for the page counts.
me too. I have complained and haven't gotten anyone to figure this out. I also feel allergic to PHP and haven't wanted to try and decode that.
Heh... considering the other subject at hand, I believe that PHP can call a stored procedure. 😀
Anyway, thanks for the feedback Steve. I appreciate it.
--Jeff Moden
Change is inevitable... Change for the better is not.
Viewing 15 posts - 66,286 through 66,300 (of 66,816 total)
You must be logged in to reply to this topic. Login to reply