Big Data

Do You Have Big Data?

  • Editorial

Data sizes are always growing. Stats on world data are astounding, as are the stats many of us experience in our lives. Plenty of us have moved from MB management to GBs, and I see plenty of people dealing with TB storage at home. Most of that data is likely from images and video, but […]

2020-09-09

130 reads

SQL Server Integrates Hadoop and Spark out-of-the box: The Why?

  • Article

Introduction Microsoft announced in September 2018 that SQL Server 2019, which is now in preview, will have a Big Data Cluster deployment option. This is a Big-Data-capable SQL Server with elastic scale and extended Artificial Intelligence (AI) capabilities, mostly as a result of deep integration of Hadoop and Spark out-of-the-box.  The new SQL Server Big […]

4.67 (6)

2019-09-09

5,677 reads

Why Would I Ever Need to Partition My Big ‘Raw’ Data?

  • Article

Whether you are running an RDBMS, or a Big Data system, it is important to consider your data-partitioning strategy. As the volume of data grows, so it becomes increasingly important to match the way you partition your data to the way it is queried, to allow 'pruning' optimisation. When you have huge imports of data to consider, it can get complicated. Bartosz explains how to get things right; not perfect but wisely.

2016-11-22

3,345 reads

How to Start Big Data with Apache Spark

  • Article

It is worth getting familiar with Apache Spark because it a fast and general engine for large-scale data processing and you can use you existing SQL skills to get going with analysis of the type and volume of semi-structured data that would be awkward for a relational database. With an IDE such as Databricks you can very quickly get hands-on experience with an interesting technology.

2016-11-18

3,131 reads

The End of Big Data

  • Article

What is next for big data? Some experts claim that data "volumes, velocity, variety and veracity" will only increase over time, requiring more data storage, faster machines and more sophisticated analysis tools. However, this is short-sighted, and does not take into account how data degrades over time. Analysis of historical data will always be with us, but generation of the most useful analyses will be done with data we already have. To adapt, most organizations must grow and mature their analytical environments. Lockwood Lyon shares the steps they must take to prepare for the transition.

2016-06-03

10,764 reads

Blogs

Custom deployment parameters in Azure Data Factory - Forget Me Not

By

(2020-Sep-26) Last week one of my Azure Data Factory (ADF) deployment pipelines failed with an...

VMworld 2020 – Top 10 Database Sessions (Part 2)

By

Whether you are a SQL Server Database Administrator, Infrastructure Architect, or any other member...

Daily Coping 25 Sep 2020

By

I started to add a daily coping tip to the SQLServerCentral newsletter and to...

Read the latest Blogs

Forums

Report Server Load balanced issue with one server while accessing https:

By PJ_SQL

We have 2 servers which are load balanced, when reporting service is stopped on...

multipoint to line geometry

By lpro

Hi, Attached, I exported a CSV file from a multipoint record from sql server ...

Using SSIS packages build in VS2010 when migrating SQL server from 2008 to 2016

By blom0344

With eof of Windows2008R2, lots of our customers are planning new server setups, usually...

Visit the forum

Ask SSC

SQL Server Q&A from the SQLServerCentral community

Get answers