Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase 12»»

Amazon Cloud Storage Expand / Collapse
Author
Message
Posted Monday, May 31, 2010 9:48 AM
SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: Monday, November 17, 2014 12:23 PM
Points: 211, Visits: 521
Hello Folks

I am about to work with Designing n Maintaining DBS in a company which is using Amazon Cloud Storage. So i was wondering how different it is from using a External HDD or Network Drive to store and retrieve Data from cloud storage.

I wish it is as simple as placing the data files on a Network Drive... is it so ?

What is the exact interface which we use to access the data from that web service ?

Will cloud services be able to provide the fast data access for fetching queries ?

IS internet speed a major factor that decides the speed of my DB ?

Its a pretty small company with all in one (DB design, developer n Admin) so suggestions are welcome.






Today is the tomorrow you worried about yesterday
Post #930412
Posted Monday, May 31, 2010 11:31 AM


SSC-Insane

SSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-InsaneSSC-Insane

Group: General Forum Members
Last Login: Thursday, November 20, 2014 8:50 PM
Points: 20,584, Visits: 9,623
How about using sql express 2008 [advanced] on a local machine? If you really have only a few users, that'd be plenty strong enough and you wouldn't have to worry about amazon integration or anything going wrong with the internet or amazon.
Post #930446
Posted Monday, May 31, 2010 5:34 PM


SSC-Dedicated

SSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-Dedicated

Group: Administrators
Last Login: Today @ 7:12 AM
Points: 31,284, Visits: 15,746
AFAIK, you can't run SQL Server locally and put the files on Amazon's cloud service. You can contract with them for database services, and in that case, the speed of the database will depend on the network as well as individual machine speed.







Follow me on Twitter: @way0utwest

Forum Etiquette: How to post data/code on a forum to get the best help
Post #930488
Posted Tuesday, June 1, 2010 1:44 AM
Grasshopper

GrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopper

Group: General Forum Members
Last Login: Monday, April 22, 2013 9:55 PM
Points: 10, Visits: 61
I don't really get what you mean. They are using S3 for storage or they want to have the database in EC2.

If the former then you can forget it sorry. You can use S3 to back up the DB though.

If the latter then that's fine and is pretty easy to set up. Be aware that I/O will be worse than you're used to so factor that into planning.
Post #930565
Posted Tuesday, June 1, 2010 2:00 PM
SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: Monday, November 17, 2014 12:23 PM
Points: 211, Visits: 521
@@Steve

I am using EC2.
Why cannot i run SQL Server Locally and put my DB files on Cloud?
So how can SQL Server and Cloud storage work together?
Reference to any articles and info tin this regard is welcome

Thank You



Today is the tomorrow you worried about yesterday
Post #930964
Posted Tuesday, June 1, 2010 2:07 PM


SSC-Dedicated

SSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-Dedicated

Group: Administrators
Last Login: Today @ 7:12 AM
Points: 31,284, Visits: 15,746
SQL Server doesn't map the data files to a network location by default, and it won't map to most transfer protocols and work at any reasonable speed. It really requires very high speed access, at bus speeds, or SAN/HBA speeds, not network speeds. Most times people have issues when they put database files on their local network.

You're asking for trouble trying to do this. Contract with Amazon to run the instance in their cloud, or keep it local.







Follow me on Twitter: @way0utwest

Forum Etiquette: How to post data/code on a forum to get the best help
Post #930966
Posted Tuesday, June 1, 2010 2:49 PM


Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Tuesday, January 7, 2014 2:09 PM
Points: 61, Visits: 237
What are you trying to accomplish? Is this for DR, high availability or some other reason?

And how much space do you think you will need? Sorry for all the questions, just trying to get a better idea what your needs are.
Post #930985
Posted Tuesday, June 1, 2010 2:55 PM
Grasshopper

GrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopper

Group: General Forum Members
Last Login: Monday, April 22, 2013 9:55 PM
Points: 10, Visits: 61
Firstly, I agree with Steve.

Having said that, if he absolutely must run this in S3 then I've thought of a way that might work. Jungle Disk (www.jungledisk.com) presents a S3 volume as a local volume and uses caching to give good performance. I have my home directory on Jungle Disk and it's quite acceptable, you never really notice that things are being backed up transparently. Whenever I sit at another computer my files are 'just there', whether that computer is elsewhere in the office or elsewhere in the world. It's pretty cool.

That's files... a database, well, Jungle Disk won't know about SQL transactions and I would be stunned to find the remote copy is guaranteed to be in a correct state at all times. Also, I don't know how SQL Server adds data to a file or how Jungle Disk copies it but I'd be surprised if Jungle Disk elegantly handled keeping a many-gigabyte file stored. It might work I suppose... just like running a data warehouse in SQLite might work too :)

I think you should go back to whoever the IT manager is at this company and tell them that SQL just doesn't run without fast local storage and you're happy to use S3 for all backups (daily, hourly, etc.) but you can't use it for physically storing the database.
Post #930988
Posted Tuesday, June 1, 2010 9:44 PM


SSC-Dedicated

SSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-DedicatedSSC-Dedicated

Group: Administrators
Last Login: Today @ 7:12 AM
Points: 31,284, Visits: 15,746
One thing to be wary of with remote storage is that SQL doesn't know things are committed, and you could end up with an inconsistent database, especially if the transfer from your local disk to Amazon went down. I know some people that have had this happen with SANs when the SAN had too much stuff written to cache, and SQL was expecting those transactions hardened to disk. If the power drops or there's some issue and the data never gets written to disk, you'll restart SQL in an inconsistent state.

SQL is not recommended for remote storage. The disks must appear local, and I'm not sure this would be a supported config if you had issues and called MS.







Follow me on Twitter: @way0utwest

Forum Etiquette: How to post data/code on a forum to get the best help
Post #931080
Posted Thursday, June 3, 2010 10:04 AM
SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: Monday, November 17, 2014 12:23 PM
Points: 211, Visits: 521
<b>[color=#FF0000]@@Dave Jones[/color]</b>

We are looking for high availability and scalability, and by using EC2 we can be less worried about buying our own servers etc.. Since ours is a small business..What all we need is
1.Data being quried out of DB via website
2.Data being written into DB via forms on website
3.Export batches/2 or 3 excel files into few tables
Thats all ..
No Log shipping
Replications
Automated Jobs etc

<b>[color=#FF0000]@@Steve Jones[/color]</b>

Thank You Steve. SO what you said is in reference to S3 storage right ?
As i told you we are using EC2 which will have its own virtual servers.
So would that still be a negative aspect to what we are trying to accomplish.

<b>[color=#FF0000]@@Corins[/color]</b>

thank You cornis. Jungle-disk is cool. But my problem is
1. My managements asked me to design a DB
2. then integrate it to Web and
3. Once all design is set .. They want to move it to cloud

so that they have less burden of managing the servers..

So do they still need a DBA after Step 2 ?
I don't want myself to be laid off :P




Today is the tomorrow you worried about yesterday
Post #932264
« Prev Topic | Next Topic »

Add to briefcase 12»»

Permissions Expand / Collapse