The Invisible Database

As we enter 2012, our engineering team is hard at work on our new SQL Anywhere OnDemand Edition “Fuji” product that is currently planned for GA launch in the 1st half of this year. Everyone is involved, for example:

• Development is hard at work on the last set of features and capabilities, fixing software issues and bugs, testing performance and scalability
• QA is working closely with development to create and run a whole series of new test suites
• The Doc team is busy writing new books to describe how to use the new software
• Other folks are busy working out the details for a new cloud-friendly payment process

As you read through this list, you may recognize elements of your own engineering organization. Do we have other matters to deal with? Of course we do:

• Fixing customer issues (bugs)
• Enhancements to SQL Anywhere Server, MobiLink and UltraLite in preparation for its next major release, currently scheduled in early 2013 (more about that later this year)

Our team’s goals this year are to continue consulting our customers to ensure we understand what problems are being faced so that we are in a better position to create innovative software to solve those problems. As I say this, I am reminded of a phrase I heard recently:

To understand how to make a product better, understand what job the product is being hired to perform. – Clayton Christensen as heard listening to his talk at the Business of Software 2011 conference

One of my personal goals each year is to interact with our customers, to better understand what “job” our products are being hired to perform.

As a personal note, I would like to thank you for taking time from your busy schedules to read my blog. Andrew Coyne, a columnist in one of the newspapers I read, recently posted this thought, which resonated with me:

Of necessity, then, the writer who wishes to be read must begin with an attitude of humility before the reader. Your parents told you nobody owes you a living? Nobody owes you the two minutes it takes to read your column.

As I start this new year of 2012, I hope to keep you informed about what our team is doing, how we do it, and most importantly, to try and provide you with information you can use in your own organization to make a difference.

On behalf of all of us on the SQL Anywhere team, I would like to wish each and everyone of you a Happy New Year!

Since the launch of our new SQL Anywhere OnDemand Edition (code named “Fuji”), I have observed that the phrase “Multi-tenant database” is most often used by other vendors to refer to the “Shared Schema” approach I described in Part 5 of my series on Multi-Tenant Database Architectures published in 2009 (1, 2, 3, 4, 5). The important thing I would like to ensure you realize is that there ARE other options. First, lets revisit the challenges that the shared schema model presents to the application developer and the DBA alike:

• Application Development Time can be much longer because each and every database access must ensure that only one customer’s data is being accessed. Some cloud vendors have tried to work around this limitation by implementing an entire proprietary development system on top of their database server. In these systems, developers must use the vendor system, rather than their favorite application development tool.
• Database Performance Tuning can be much more difficult, or even impossible, because every database table has data from many different customers, and that data may be accessed in very different ways by each customer. Additionally, the proprietary development environments provided by some vendors completely eliminate any opportunity for database tuning.
• Security can be problematic. The application or the proprietary environment itself implements the security, rather than the database server. The application must be very robustly tested to ensure that one customer does not see another customer’s data. A minor application bug could completely compromise data security.
• Customization can be very difficult or impossible. All tenants share the same schema, so every customer’s application needs to be virtually identical. The use of generic schema setups may enable some simple customization, such as the addition of a few extra fields, but even this is limited by the generic schema system. Large scale customization would be virtually impossible.

Even Sybase and SAP’s rival Larry Ellison from Oracle has got into the action in October, criticizing the Shared Schema approach as implemented by SalesForce.com:

That’s a very bad security model. It’s called multi-tenancy and it was state of the art 15 years ago. This is 2011. All the modern compute clouds use virtualisation as part of their security model. You get a separate virtual machine, your data’s in a separate database because it’ virtualised. They put your data at risk by commingling it with others.

Salesforce.com’s database intermingles one everyone’s data, and relies on application system logic to ensure data security.

Our new SQL Anywhere OnDemand Edition uses a completely different approach than the Shared Schema model. It implements a Shared Server, Separate Database architecture for handling multiple tenants. This IS the virtualised database system referred to by Mr. Ellison.

The Shared Server, Separate Database model has many advantages, which I will explore in greater detail in my next post.

Posts, not years, that is. Without even realizing the milestone attached to it, my previous post Behind Every “Cloud” There are Real People was the 100th post to the Invisible Database blog. When my 4 colleagues (Glenn, Eric, Jason and Tom) and I started our blogs in 2008, we were among the first bloggers in Sybase. Today, there are many Sybase sponsored blogs, all accessible from the Sybase Blog Center.

Over the couple of years of blogging, I’ve personally experienced the “Dip” talked about by Seth Godin, where the fun and excitement of starting something new fades into the hard work of constantly thinking about the next post, and making it interesting and beneficial for you, the reader.

One of my favorite early posts described my choice for the blog name Why Invisible Database? Over the last couple years, I have become even more convinced that “The SQL Anywhere engineering team is, in my opinion, one of the most talented set of individuals ever put together.” Our current effort to create SQL Anywhere OnDemand has stretched our team’s imagination about what could be, rather than what has already been done by others.

The toughest post to write was my post at the beginning of this year where I described our team’s loss of my long time friend and colleague Geno Coschi.

Thank you for reading. I look forward to continue sharing insights from the SQL Anywhere team.

Chris

To many of us today, the “cloud” seems like a bit of magic. We often simply use the services of a cloud based system without really thinking about where the cloud is located, or who keeps it running. Ultimately, behind every “cloud”, there are real people managing real machines. What is marketed as a “cloud” is really a rack of machines, with a very real person who has to keep them running. To that person; the administrator, the “cloud” isn’t “in the cloud”, it’s in his own data center! The administrator must put together a set of machines, software and administrative tools that enable everything to be viewed in a completely hands-off way by the users, so that they think of it as a “cloud”.

The challenge vendors face when trying to market and sell something for “the cloud” is that the definition of “the cloud” is so broad and varied. What passes as the cloud to one person is simply a set of machines to another person. Our new SQL Anywhere OnDemand Edition (code named “Fuji”) currently in beta test, is one such product designed to help the administrator of those machines to create a data cloud. While the administrator certainly will know what machines are in use, where the database servers are running and where the databases are located, the end user will be enabled to simply view their database as “being in the cloud”.

The administrator of the cloud system will use one of the primary components of Fuji: the administrative console. The console is designed to enable an administrator to easily keep track of the various “host” machines that are part of the system, the SQL Anywhere servers that are running on each host, and the databases being served by each database server. The console also provides access to all the various tasks that an administrator might want to execute on their running cloud; including starting and stopping database servers, adding a new database into the cloud, setting up high-availability for a database, and backup/restore operations. The console is completely web-driven, so it can be accessed using a standard Flash-enabled web browser.

You can find out more about Fuji and try it out yourself. Take part in the Beta program by signing up on the Fuji Beta Web site.

Fuji is now released into beta!

Fuji is the code name for our new SQL Anywhere OnDemand Edition product, designed to enable easy management of multi-tenant, hosted applications by Independent Software Vendors (ISVs). We announced Fuji in September at our Techwave 2011 user conference, indicating in the press release that the software would be available within 30 days. Yesterday was exactly 30 days, and the software was actually available yesterday, on target, while still meeting our quality goals for the beta!

Over the last couple years, I have had the wonderful opportunity to speak with many of our ISV partners. Many of them have told be about their customers. Increasingly, many of their customers are asking if the ISV has a hosted/cloud offering. SQL Anywhere OnDemand Edition is designed to enable these ISVs to more easily host multi-tenant database applications, and offer this as an additional product offering for some of their customers.

I’ve written about multi-tenant database architectures in the past. The 5 part series discussed several different approaches for handling data isolation and scalability. (Part 1, Part 2, Part 3, Part 4, and Part 5) Fuji implements the “shared database server, separate database” model described in Part 3.

As I noted in Part 3, this model has several advantages:

• Development time – There is minimal additional development time when compared to an on-premise solution.
• Hardware cost – This architecture allows for improved utilization of machine memory resources, however still requires overhead for unique databases per tenant.
• Application and database performance – The performance of one tenant may be impacted by the activities of other tenants sharing the server machine.
• Security – Each tenant has their own database, completely isolating them from other tenants.
• Customization requirements – Each tenant has their own database, so it is easy to customize the database schema for the differing needs of each tenant.
• The number of tenants – Able to handle more tenants than the separate server approach, but still limited based on the need to administer and store separate database files for each tenant.

The big advance in Fuji is that easy management of many servers, each running several databases, all easily managed from a single console.

You can find out more about Fuji, and take part in the Beta program by visiting the Fuji Beta Web site.

Earlier this week I was directed to an essay, written over 10 years ago by Paul Graham, called “Being Popular“. Graham is writing primarily about computer languages, however I think his observations in section 10 of the essay are equally appropriate to any software development project.

To write good software you must simultaneously keep two opposing ideas in your head. You need the young hacker’s naive faith in his abilities, and at the same time the veteran’s skepticism. You have to be able to think how hard can it be? with one half of your brain while thinking it will never work with the other.

Graham goes on to observe:

People who do good work often think that whatever they’re working on is no good. Others see what they’ve done and are full of wonder, but the creator is full of worry. This pattern is no coincidence: it is the worry that made the work good.

and

It’s tricky to keep the two forces balanced. In young hackers, optimism predominates. They produce something, are convinced it’s great, and never improve it. In old hackers, skepticism predominates…

I think these observations sum up exactly what is happening on our team right now as we work to release our new Fuji product into beta. Our team is a good mix of people, encompassing some of us with many years of product release experience, with many younger folks, demonstrating nearly unbounded enthusiasm.

We are truly excited by the possibilities that we envision for the future of multi-tenant hosted/cloud database systems, yet the latest P1 issue that “must be fixed before release” is weighing on our minds.

As I write this, the software is coming together nicely, and we are on track to release the beta shortly. You can find out more about Fuji, and take part in the Beta program by visiting the Fuji Beta Web site.

I would like to relate to you a true story. Two weeks ago, our annual user conference, Techwave 2011, occurred in Las Vegas, co-located with SAP TechEd. I was scheduled to deliver 2 different sessions on SQL Anywhere, as well as participate in the Data Management Plenary session. I had created my PowerPoint slides, along with my demos in advance. As I left the office on Friday, prior to flying out, the only thing remaining to do was practice and fine tune my section in the Data Management Plenary session. My plan: Work on this during the 4 hour flight.

While waiting at the airport, and on the flight, I pulled out my computer to work on my talk. I was only given 15 minutes to speak, so I had to ensure that I covered the right points, and didn’t allow myself to get side-tracked. I do take this preparation time seriously. I read a quote recently, reportedly from former US President Woodrow Wilson:

If I am to speak ten minutes, I need a week for preparation; if fifteen minutes, three days; if half an hour, two days; if an hour, I am ready now.

At some point during the flight after an hour or so of scripting work typed in, but unsaved, my computer screen went haywire, flashing like a strobe light, followed 10 or 15 seconds later by the dreaded Blue Screen of Death.

YUP, my computer had just died on my way to our week long user conference! Was I annoyed? You bet I was. (You might imagine that I actually thought some stronger words )

I had my trusty BlackBerry, so I would still have access to email. Then, I remembered that on Friday, September 9, my last day in my office just before leaving for the weekend, I realized that I didn’t have a backup of my demos. I had spent 30 or 40 minutes clearing enough room on a USB stick, and copying my PPTs and demos over to the stick, JUST IN CASE.

Why did I do that? Over the years, we have established a general best practice on the SQL Anywhere team that you should always make sure there is a backup of your talks. It can be on a USB stick, ensuring that someone else has your TechWave talks, or that they be sent in email to other folks on the team, JUST IN CASE. In my case, my talks had already been sent to the TechWave organizers, AND I had them on 2 different USB sticks.

What did I do?
Since I was unable to use my own computer, I copied my talks and demos from my USB drive over to Jason Hinsperger’s computer. I used his machine for my talk and demo. Everything went fine. My backup came to the rescue.

The moral of the story? BACKUPS ARE IMPORTANT. DON’T OVERLOOK THEM!

PS:
When I returned to Waterloo, and upon further diagnosis, we discovered that my laptop’s screen had died. My hard drive was intact, so we simply popped it into a similar laptop, and I was up and running again within a day.

Here is the question: As a technology professional, who lives and breathes in the fast paced technology industry, how many times does something actually make you stop and say “Wow, is that ever cool!”? Perhaps some new device paradigm such as the tablet, or some very interesting new piece of software. I’m going to be honest, having worked on databases for close to 20 years, very rarely anymore am I truly caught off-guard by anything new in the data management industry.

That standard fell this past summer when I started playing with preliminary builds of an innovative set of software tools designed to enable ISVs to more easily create and deploy databases in multi-tenant “cloud” like environments. I’ve written about using SQL Anywhere for ISV clouds several times this year, so, it should be no surprise that we are indeed working on something to address this space.

Announced at our Techwave 2011 user conference 2 weeks ago, our new product, code named “Fuji”, is one of the coolest ideas I have seen in a long time. Fuji allows anyone to setup and manage a large number of tenant databases in a hosted / cloud environment. I think this will be especially valuable to our ISV partners who are more and more hearing requests from their customers to deliver a hosted or cloud solution. Fuji offers many advantages, including the ability of a software developer to support ONE database solution for BOTH their on-premise customers, as well as their hosted customers.

I will be writing more about Fuji throughout the fall.

You can find out more about Fuji, and take part in the Beta program by visiting the Fuji Beta Web site.

Seth Godin writes what I believe is a quite popular blog about various marketing and business topics. One of the interesting things, from my point of view is that he posts EVERY single day. Usually, his posts are small (2 or 3 paragraphs), occasionally tinier, and sometimes much longer. Almost always, they are thought provoking. Reading his blog has become part of my morning routine, usually after I get my coffee, while I wait for my machine to start up. I only spend a minute or two reading his daily post, but often the thoughts he has provoked last much longer. If you don’t want your current approach to business to be challenged, then you should avoid his blog.

Last Friday, Godin’s post was called “Talker’s Block“. His premise is that no one has nothing to say, and the reason is practice. Writers, on the other hand claim to suffer from “writer’s block”. Why? In his mind it is because they don’t practice enough. His recommendation is to write often, and in the public. Only through this constant exercise will someone become a better writer.

Clearly, Godin, a prolific writer of books as well as his blog, is urging those who blog, such as myself, to write more often. This seems like sound advice as we head into the fall.

Chris

Today, August 12, our SQL Anywhere web forum at http://sqlanywhere-forum.sybase.com received it’s 1000th question.

Breck Carter started this forum and has shepherded it through it’s many growing pains, and continues to be very active. Thank you Breck!

Some interesting stats on the top non-Sybase users of the forum:

16 non-Sybase users on 1st user page of 35
The top 3 users by reputation:

1. Volker Barth, with a reputation of 8142
2. Breck Carter – 3884
3. Martin – 2286

Demonstrating that the forum is a world-wide community, the following countries are represented on 1st page of users:

• Germany
• Canada
• Brazil
• United States
• United Kingdom
• Russia

As you can imagine, it also takes efforts from a variety of folks inside the SQL Anywhere team to keep the forum running, and provide many of the answers that could only be given by “the person who wrote the code.” These include “first line’rs”:

• Mark Culp
• Glenn Paulley
• Karim Khamis

and

• Graeme Perrow – who maintains the forum software

Congratulations to everyone involved with the forum!

If you haven’t had a chance, check out the forum at:

http://sqlanywhere-forum.sybase.com.