Click here to monitor SSC
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Amazon Cloud Storage


Amazon Cloud Storage

Author
Message
Minnesota - Viking
Minnesota - Viking
SSC Veteran
SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)

Group: General Forum Members
Points: 217 Visits: 564
Hello Folks

I am about to work with Designing n Maintaining DBS in a company which is using Amazon Cloud Storage. So i was wondering how different it is from using a External HDD or Network Drive to store and retrieve Data from cloud storage.

I wish it is as simple as placing the data files on a Network Drive... is it so ?

What is the exact interface which we use to access the data from that web service ?

Will cloud services be able to provide the fast data access for fetching queries ?

IS internet speed a major factor that decides the speed of my DB ?

Its a pretty small company with all in one (DB design, developer n Admin) so suggestions are welcome.


Today is the tomorrow you worried about yesterday:-)

Ninja's_RGR'us
Ninja's_RGR'us
SSC-Insane
SSC-Insane (20K reputation)SSC-Insane (20K reputation)SSC-Insane (20K reputation)SSC-Insane (20K reputation)SSC-Insane (20K reputation)SSC-Insane (20K reputation)SSC-Insane (20K reputation)SSC-Insane (20K reputation)

Group: General Forum Members
Points: 20931 Visits: 9671
How about using sql express 2008 [advanced] on a local machine? If you really have only a few users, that'd be plenty strong enough and you wouldn't have to worry about amazon integration or anything going wrong with the internet or amazon.
Steve Jones
Steve Jones
SSC-Dedicated
SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)

Group: Administrators
Points: 36060 Visits: 18736
AFAIK, you can't run SQL Server locally and put the files on Amazon's cloud service. You can contract with them for database services, and in that case, the speed of the database will depend on the network as well as individual machine speed.

Follow me on Twitter: @way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
corrin.lakeland
corrin.lakeland
Grasshopper
Grasshopper (16 reputation)Grasshopper (16 reputation)Grasshopper (16 reputation)Grasshopper (16 reputation)Grasshopper (16 reputation)Grasshopper (16 reputation)Grasshopper (16 reputation)Grasshopper (16 reputation)

Group: General Forum Members
Points: 16 Visits: 68
I don't really get what you mean. They are using S3 for storage or they want to have the database in EC2.

If the former then you can forget it sorry. You can use S3 to back up the DB though.

If the latter then that's fine and is pretty easy to set up. Be aware that I/O will be worse than you're used to so factor that into planning.
Minnesota - Viking
Minnesota - Viking
SSC Veteran
SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)

Group: General Forum Members
Points: 217 Visits: 564
@@Steve

I am using EC2.
Why cannot i run SQL Server Locally and put my DB files on Cloud?
So how can SQL Server and Cloud storage work together?
Reference to any articles and info tin this regard is welcome

Thank You


Today is the tomorrow you worried about yesterday:-)

Steve Jones
Steve Jones
SSC-Dedicated
SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)

Group: Administrators
Points: 36060 Visits: 18736
SQL Server doesn't map the data files to a network location by default, and it won't map to most transfer protocols and work at any reasonable speed. It really requires very high speed access, at bus speeds, or SAN/HBA speeds, not network speeds. Most times people have issues when they put database files on their local network.

You're asking for trouble trying to do this. Contract with Amazon to run the instance in their cloud, or keep it local.

Follow me on Twitter: @way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
Dave Jones - CRC
Dave Jones - CRC
Valued Member
Valued Member (63 reputation)Valued Member (63 reputation)Valued Member (63 reputation)Valued Member (63 reputation)Valued Member (63 reputation)Valued Member (63 reputation)Valued Member (63 reputation)Valued Member (63 reputation)

Group: General Forum Members
Points: 63 Visits: 239
What are you trying to accomplish? Is this for DR, high availability or some other reason?

And how much space do you think you will need? Sorry for all the questions, just trying to get a better idea what your needs are.
corrin.lakeland
corrin.lakeland
Grasshopper
Grasshopper (16 reputation)Grasshopper (16 reputation)Grasshopper (16 reputation)Grasshopper (16 reputation)Grasshopper (16 reputation)Grasshopper (16 reputation)Grasshopper (16 reputation)Grasshopper (16 reputation)

Group: General Forum Members
Points: 16 Visits: 68
Firstly, I agree with Steve.

Having said that, if he absolutely must run this in S3 then I've thought of a way that might work. Jungle Disk (www.jungledisk.com) presents a S3 volume as a local volume and uses caching to give good performance. I have my home directory on Jungle Disk and it's quite acceptable, you never really notice that things are being backed up transparently. Whenever I sit at another computer my files are 'just there', whether that computer is elsewhere in the office or elsewhere in the world. It's pretty cool.

That's files... a database, well, Jungle Disk won't know about SQL transactions and I would be stunned to find the remote copy is guaranteed to be in a correct state at all times. Also, I don't know how SQL Server adds data to a file or how Jungle Disk copies it but I'd be surprised if Jungle Disk elegantly handled keeping a many-gigabyte file stored. It might work I suppose... just like running a data warehouse in SQLite might work too Smile

I think you should go back to whoever the IT manager is at this company and tell them that SQL just doesn't run without fast local storage and you're happy to use S3 for all backups (daily, hourly, etc.) but you can't use it for physically storing the database.
Steve Jones
Steve Jones
SSC-Dedicated
SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)SSC-Dedicated (36K reputation)

Group: Administrators
Points: 36060 Visits: 18736
One thing to be wary of with remote storage is that SQL doesn't know things are committed, and you could end up with an inconsistent database, especially if the transfer from your local disk to Amazon went down. I know some people that have had this happen with SANs when the SAN had too much stuff written to cache, and SQL was expecting those transactions hardened to disk. If the power drops or there's some issue and the data never gets written to disk, you'll restart SQL in an inconsistent state.

SQL is not recommended for remote storage. The disks must appear local, and I'm not sure this would be a supported config if you had issues and called MS.

Follow me on Twitter: @way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
Minnesota - Viking
Minnesota - Viking
SSC Veteran
SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)SSC Veteran (217 reputation)

Group: General Forum Members
Points: 217 Visits: 564
<b>[color=#FF0000]@@Dave Jones[/color]</b>

We are looking for high availability and scalability, and by using EC2 we can be less worried about buying our own servers etc.. Since ours is a small business..What all we need is
1.Data being quried out of DB via website
2.Data being written into DB via forms on website
3.Export batches/2 or 3 excel files into few tables
Thats all ..
No Log shipping
Replications
Automated Jobs etc

<b>[color=#FF0000]@@Steve Jones[/color]</b>

Thank You Steve. SO what you said is in reference to S3 storage right ?
As i told you we are using EC2 which will have its own virtual servers.
So would that still be a negative aspect to what we are trying to accomplish.

<b>[color=#FF0000]@@Corins[/color]</b>

thank You cornis. Jungle-disk is cool. But my problem is
1. My managements asked me to design a DB
2. then integrate it to Web and
3. Once all design is set .. They want to move it to cloud

so that they have less burden of managing the servers..

So do they still need a DBA after Step 2 ?
I don't want myself to be laid off Tongue


Today is the tomorrow you worried about yesterday:-)

Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search