July 14, 2026 at 3:57 am
Hi,
We are looking out to read parquet file directly from on premise shared network path. The purpose is to read the data and move it to staging table and thereafter the etl process will remain as existing. To what I read online that we can read parquet files directly using openrowset(…bulk). There will be hundreds of files across 4 tables and for each tables separate folder shall be maintained.
We are not opting for poly base as the files will be required to be stored in object storage and we are bound by on premise setup. (Not even MinIO)
Hence experts kindly give your suggestion.
July 14, 2026 at 9:26 am
Also interested as my data analyst considers parquet files as datasource for the datawarehouse using python / duckdb
July 14, 2026 at 4:49 pm
Is this S3 storage? Or Azure blob?
AFAIK, with OpenRowset in SQL Server 2022 requires polybase to do this. I don't think there is a provider that enables this without PB.
Example from MSLearn: https://learn.microsoft.com/en-us/sql/relational-databases/polybase/polybase-virtualize-parquet-file?view=sql-server-ver17
July 20, 2026 at 4:03 am
One correction : OPENROWSET(BULK ... FORMAT='PARQUET') needs the PolyBase feature installed and enabled on the SQL Server instance, full stop, regardless of where the files live. That's not the same thing as needing object storage.
PolyBase has a standalone mode that only reads local disk or UNC network paths. No S3/ADLS/MinIO, no CREATE EXTERNAL DATA SOURCE, no Hadoop connectivity config, nothing external at all. So "we're on-prem and won't use object storage" doesn't actually rule this out on its own, those are two separate concerns that got conflated.
Effect: it's a real, standing footprint change on a production instance (extra services, extra memry, one more thing to patch and monitor) for what is functionally just a file reader.
The alternative is a small custom loader (PowerShell or .NET, e.g. Parquet.Net + SqlBulkCopy, or Python with pyarrow) that enumerates each folder, loads new Parquet files into the staging tables, and leaves SQL Server completely untouched, no new services, no restart. It's more code up front, but with hundreds of files across 4 folders you'll want per-file tracking (processed/failed, retries, move-to-done) regardless of which approach you pick, and OPENROWSET gives you none of that bookkeeping for free either way.
OPENROWSET is less code but adds a standing service footprint to the instance. A script-based loader is more code but keeps the SQL Server surface unchanged and gives full control over error handling per file.
Viewing 4 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply