NoSQL Datastores of Interest to the .NET Developer

The world of NoSQL is vast, and this is in no way a comprehensive list of NoSQL datastores (just see http://en.wikipedia.org/wiki/NoSQL for how vast the NoSQL universe is).  After spending a lot of time researching, this is just a list of ones that have officially supported C# libraries, or are written in .NET.  I thought it would be a little easier to start learning the ins-and-outs of the different types of datastores without having to learn new languages also.  Pretty much every NoSQL datastore has some sort of REST-ful API, which you can work with regardless of your language choice.  Don’t let the presence or absence of a system on this list make your decision for your application choices–this is more a list of systems I think would be fairly simple and interesting to experiment with.  I have not yet worked with all of these datastores, but as I do I’ll add posts to this blog.

Every category of NoSQL is designed to solve a particular problem, and each option in each category has its ups and downs.  There are many options, so you really need to know what you want to do.  Do you need an embedded solution, or a scalable cluster?  Are you trying to discover relations between populations, store profile data in a flexible schema, or cache the results of API requests?  What types of indexing are supported?  Can you live with eventual consistency?

I tried to note some easy and low cost ways to get started with each datastore.  If there isn’t direct DBaaS support, you can always deploy Azure or AWS VMs, and even the Google Cloud Platform has some hosting options.  Try not to install everything on your local machine, but have fun!

Category Datastore .NET Support Notes
Graph neo4j http://neo4j.com/developer/dotnet/ Neo4j is perhaps the best known graph database.  Although the clients are community supported, they are maintained by two amazing developers.  It’s easy to get started, especially since GrapheneDB offers a free Hobby account.  There is also a simplified Azure VM deployment.
Titan none This is on the list as “something to watch”.  Datastax (Cassandra) recently acquired the company behind Titan, and Datastax has a good history of .NET support.
OrientDB https://github.com/orientechnologies/OrientDB-NET.binary There are also community supported clients.  OrientDB is a hybrid datastore, supporting both document and graph features.
VelocityGraph Written in C# An open source hybrid (graph/document) datastore, written in C# which can be embedded or distributed.  There is a paid model for the distributed version also.
Document MongoDB Official and community clients listed at http://docs.mongodb.org/ecosystem/drivers/csharp/ One of the best known and most used document datastores, MongoDB is backed by 10gen, and mongolab.com offers a free sandbox account to get started.  MongoDB and Mongolab are available via Azure Marketplace, so you can spend free Azure credits if you have them.
Azure DocumentDB It’s Microsoft, no worries.
https://msdn.microsoft.com/en-us/library/dn781482.aspx		 This is still in Preview at the time of writing, but it looks very promising.  Being Azure, if you have Azure credits you can spend them on this.  It’s another DBaaS, so you won’t need to mess with VMs.
Amazon DynamoDB http://aws.amazon.com/sdk-for-net/ Another high performance DBaaS datastore, DynamoDB supports both document and key-value modes.  This is included in AWS’s free tier for a year.
OrientDB https://github.com/orientechnologies/OrientDB-NET.binary There are also community supported clients.  OrientDB is a hybrid datastore, supporting both document and graph features.
VelocityGraph Written in C# An open source hybrid (graph/document) datastore, written in C# which can be embedded or distributed.  There is a paid model for the distributed version also.
CouchDB https://couchdb.apache.org/ Another popular datastore, this is the open source project from Apache.  This is supported by Couchbase (see below)
Couchbase http://docs.couchbase.com/couchbase-sdk-net-1.2/ A next generation of CouchDB, in a way (see http://www.couchbase.com/couchbase-vs-couchdb).  Has both open source and commercial licenses.  Popular as an in-memory cache by some big name companies.  There is an Azure VM image in the Azure VM Depot.
RavenDB Written in .NET Open source and commercial licenses.  Has a embedded and scalable server options.  A RavenHQ hosted plan is available through the Azure Marketplace.
NinjaDB Pro Written in .NET Commercial, embeddable document datastore which is also compatible with Xamarin.  Supports either document or relational modes.  There is also a version for WinRT.
NDatabase Written in .NET An open-source in-memory object database.
Big Table Cassandra Official driver from Datastax,
https://github.com/datastax/csharp-driver		 Datastax is the company supporting Planet Cassandra and largely supporting the Apache Cassandra project.  Datastax provides the commercial licensing for Cassandra.  Cassandra is very similar to HBase, but because of Datastax’s backing is the better choice, IMO.  Cassandra can be run on Azure (DataStax Guidance for Azure), and there is an older VM available from the Azure VM Depot.  As part of their wonderful “Succinctly” e-book series, Syncfusion also has Cassandra Succinctly.
Apache HBase community SDKs are just wrappers for the REST API.  Most of the .NET SDKs you’ll find are for HDInsight and aren’t guaranteed to work with Apache HBase. I really just put this here for comparison purposes.  HBase can be a real pain.  Seriously, look at Cassandra or HDInsight instead.
HDInsight It’s Microsoft, no worries HDInsight covers a lot of the Hadoop ecosystem, the HBase specific bits are introduced at http://azure.microsoft.com/en-us/documentation/articles/hdinsight-hbase-overview/.
Key-Value Couchbase http://docs.couchbase.com/couchbase-sdk-net-1.2/ A next generation of CouchDB, in a way (see http://www.couchbase.com/couchbase-vs-couchdb).  Has both open source and commercial licenses.  Popular as an in-memory cache by some big name companies.  There is an Azure VM image in the Azure VM Depot.
Redis There are a number of community supported clients, listed at http://redis.io/clients#c.  Two of the more popular ones are from ServiceStack and StackExchange. A very popular choice as a cache layer.  Durable persistence isn’t the strong suit, and multiple-node sharding is only in beta.  You can add Redis to Azure from the Azure Marketplace.
Azure Tables It’s Microsoft, no worries Perhaps one of the top choices, especially if the rest of your application is on Azure.  Crazy scalable and very performant.  Table storage was one of the original features of Azure, and is very well vetted by now.
Amazon DynamoDB http://aws.amazon.com/sdk-for-net/ Another high performance DBaaS datastore, DynamoDB supports both document and key-value modes.  This is included in AWS’s œfree tier for a year.
Riak https://github.com/basho-labs/riak-dotnet-client/wiki An open-source distributed datastore.  There is also a commercial offering.  An Azure VM image is available via the Azure VM Depot.

Hosting Without Limits: A Review of 1&1 Internet’s Unlimited Package

Back in the days before the ‘tubes, when all items were analog and knowledge printed, “unlimited” used to mean there were no limits. Cable companies and cell phone providers have tortured the definition to be one of fine print, term limits, caps and rate increases. When I was asked to review 1&1 Internet’s Unlimited Package (tagline: “Hosting without limits”), I entered into it with a high amount of trepidation; worried I might be reviewing “Crazy Eddie’s House of Electrons (and fine print)”. Happily, that was not the case, and I was pleasantly surprised at the ease of use, features, and price.

Earlier in my career, I developed a number of websites for small businesses and individuals. Along the way, I found myself wondering how “normal” (i.e., non-technical) people could manage. The truth was, between the complexities of hosting and the knowledge needed to build a site, they just didn’t, and that’s why they called me. Even pre-built applications meant matching requirements to the offerings of a particular host. As IT professionals, we take a lot of work upon ourselves because it’s just easier to do it ourselves than it is to explain how to do something.

Some hosting companies strive to improve the hosting experience, and over the past 27 years, 1&1 has become one of the most successful and popular hosting companies in the world by offering simple, inexpensive ($0.99/month for three months, then $8.99/month thereafter) and feature-rich hosting plans. In fact, 1&1 has over 13.5 million customer contracts and the group has more than 19 million domains registered. After working off and on for a month with my trial account, I can say this may be the hosting I could help set up for my parents or in-laws, and they could manage most things themselves.

According to the company, 1&1 has over 7,000 employees and 70,000 servers in seven data centers around the word. Beyond website hosting, 1&1 also offers domain registrations, email hosting and GeoTrust SSL certificates. This company is far more than you first imagine.

1&1’s hosting plans are all-inclusive (with unlimited space, unlimited bandwidth and unlimited sites—check that out, really unlimited!) and also include a free domain name registration. These plans can be either Windows or Linux, with MS SQL Server and MySQL as database options. Linux hosting supports PH, Perl, Python, Ruby and Zend, while Windows hosting supports PHP, .NET and Perl. These are just some of the features that appeal to the more technical user.

From the moment you sign up and first log in, you can tell right away that 1&1 is also reaching out to the less technical users. Each time you log in, a helpful tip about some aspect of your plan is displayed. 1&1 has built a custom control panel, which gives you full control over all of your services in a very simple way. If a company pays this much attention to how you interact with its services, it’s a good sign they’re paying attention to the other details of their business.

clip_image001

All new accounts are given a temporary URL so that you can begin development right away. You can upload your own code via FTP or SSH (WebDeploy does not appear to be an option), choose an application from the 1&1 App Center, or you can build a custom website using the 1&1 Website Builder and 1&1 Mobile Website Builder tools.

If you want an easy way to get a site online, and don’t mind starting from one of over 50 highly customizable layouts, the website builder tools are a great option. These tools are aimed at the Wix and SquareSpace crowd and provide an easy way to build a multi-page static website without having to know HTML, CSS or JavaScript (although you can edit these if you want to). The process is very simple: choose a page layout, select a color scheme, font and background, then add content. Create as many pages as you need, ad tweak the HTML or CSS as needed. If you find you need help with the website builders, there is a Contact button right on the menu. Live chat and phone help is available 24/7. I found the Website Builder to be simple, intuitive and complete. Again, “here you go older generation and former clients, I’ll help get you started, but you can make this happen.”

If you need a more dynamic site, or a blog, CMS or shopping cart, the 1&1 App Center features over 140 of the most popular applications, including WordPress, Joomla, Drupal and phpBB. The 1&1 App Center is a great example of how much effort 1&1 has put into simplifying the hosting experience. Every application has a detailed information page (see below), and installation requires only a couple clicks and minimal information.

clip_image003

Applications can be installed in “Safe Mode”, which is a default install of the application for which 1&1 handles all the updates and patches, or you can install an application in “Free Mode”, which gives you more flexibility, but you’re responsible for updates and patches. The Safe Mode installer is simpler than the Free Mode installer (shown below for Joomla), but if you plan on adding themes or plugins to your application, you’ll need to use Free mode. You can change a site from Safe Mode to Free Mode, but you cannot change from Free Mode to Safe Mode.

clip_image004

Regardless of how you built your site, once it’s online, it’s replicated across multiple data centers. This geo-redundancy provides failover protection, but not load balancing. Also, since the replication is nearly instant, whatever you do to destroy your main site will affect the failover before you can dial support. Fortunately, 1&1 also has daily backups of your site and data. To aid performance and security, 1&1 offers a CDN with CloudFlare traffic monitoring and Railgun™ caching, plus storage and global distribution of large libraries.

Getting yourself online is just the start. To monitor your site’s traffic, 1&1 has their own site analytics package which reports on referring sites, search engine terms and more. Additionally, there are tools to create sitemaps for Google Analytics. To keep your customers engaged, 1&1 also offers email marketing tools. Considering the cost of most email marketing services, this makes the hosting fee even more of a bargain.

For a heck of a lot of sites, I think 1&1 would be a great solution, even for the more technical crowd. It’s obvious the services and control panel are very well thought out, the services offered are feature-rich and a great value. It was pretty hard to find something to criticize, mainly that this didn’t exist 15 years ago. More advanced sites may miss uptime monitoring tools or load balancing, but anything with those needs is probably looking at a different place in the market. Although simple enough for non-technical user, there are enough features to interest a more technical crowd, especially if you have multiple domains and are paying more than $8.99/month total.

Transactional vs. ODS Talking Points

When considering implementing an operational data store, discussion always includes the differences between an ODS and a transactional database.  Transactional databases store the data for an application.  An ODS’s purpose is to consolidate from one or more transactional systems, to serve as a source of master data, or for reporting, or one source to a data warehouse.  While the purposes are pretty clear, how they differ at a design level is less clear.  Here are the talking points I’ve used in the past to describe the differences.

Transactional databases

  • are optimized for write performance and ensuring consistency of data
  • mainly inserts and updates, no table rebuilds
  • Low level of indexing, mainly primary keys and the lookups needed 
  • high use of foreign keys
  • use of history and archive tables for no longer current data
  • index and data fragmentation are a concern due to updates, and maintenance jobs need to be utilized
  • data are normalized
  • but, frequently updated data are often separated from less frequently updated data to reduce table fragmentation
  • data are raw

Operational data stores

  • are optimized for reads
  • mainly inserts and table rebuilds via ETL from transactional systems, few updates
  • high level of indexing to support querying
  • low use of foreign keys, since relations are maintained in the transactional databases
  • no history or archive tables–ODSs are for current data
  • low level of normalization, since updates are usually on the same schedule and in a batch process
  • data are sometimes calculated or rolled-up (rather than saving a birthdate, use a demographic age)
  • data may be bucketed

Exactly when to use an ODS and how the schema is designed is a discussion about balancing data duplication vs application architecture.

The update schedule of an ODS is determined partly by the needs of the ODS data consumers, and partly by what the transactional databases can tolerate.  Usually ODS updates are a batch job which runs once or several times a day.  For more frequent updates, commanding could be used.

Saving Windows RT

I consider the release of Windows RT to the consumer market to be one of the worst decisions Microsoft has made in recent years, and I have an $853MM writedown to back me up.  RT shipped primarily on a Surface RT, which isn’t an attractive personal device—it’s small, relatively costly, difficult to connect to the usual suite of peripherals and doesn’t sit well in your lap.  Additionally, here was a version of Windows which wouldn’t run any previous Windows program.  Consumers were used to getting a new computer with a new version of Windows and simply reinstalling their favorite old greeting card maker or photo editor.  Months later, when Windows 8 was released, confusion multiplied—now there were two versions of Windows—a “right one” and a “wrong one”, and your average consumer couldn’t tell the difference by looking.  Consumers literally needed someone with technical knowledge to tell the devices apart.  Add to that an a store which had few desirable apps and it’s no wonder interest was really low for RT.  The release of the Surface 3 running only Windows 8 puts the future of RT into greater doubt.

Having said that, RT could still be one of the greatest versions of Windows of all time.  How?  Improve the concept of enterprise application stores, and make RT the next Windows Embedded.  It’s not as crazy as it sounds.  I’ve helped manage installations of WinTerms for sales teams, and hundreds of handheld and lift mount devices in multiple warehouses, and this idea is a bit of a dream come true.

Windows 8 ships with a hard-coded attachment to the Microsoft store.  Make it simpler for enterprises to set up their own internal app store, and control the store setting via group policy.  Enterprises could easily distribute their in-house apps, or those supplied by ERP/WMS/etc vendors to the issued devices.  At a previous employer—a warehousing company—we had to manage hundreds of devices in multiple warehouses around the country.  We had to have someone onsite manually dock each one, and we had to go through a complicated set of steps to update the wimpy onboard apps.  If we could have posted an updated app on our internal store and have every device update itself automatically in seconds, that would have been a dream come true.  Intermec and Symbol should be all over this idea.

Take this one step further.  Remember the fires in Tesla Model S?  A software fix to correct how the car rides at freeway speed was downloaded to all the Model Ss.  Now imagine Ford replacing Sync with RT, and being able to do the same for control or entertainment systems.  Speaking of entertainment systems. keep the linkage to the movies and music stores so movies can be downloaded while parked at a McDonald’s.  The capabilities in RT would put Ford years ahead of its competitors in regards to onboard systems.  This could be extended into on-board systems for trucks as well.

Take this one more step.  Imagine battlefield updates to combat systems, downloaded via AWACs or properly equipped drones from a secure DOD app store.  It’s not too far-fetched.

Vehicles and warehouse equipment alone offers the potential of millions of devices running RT.  By looking at RT as a new Windows Embedded, Microsoft thinks big by thinking small.

An Updated Simple Passphrase Generator

(Note: The original version of this work is published at http://aspalliance.com/703_A_Simple_Passphrase_Generator.all, this is a long overdue update)

Just about 9 years ago I was building a partner-facing reporting website, and I needed a way to generate passwords when partners were added by customer service (no public registration) as well as to generate new ones easily when a password needed to be reset.  I wanted to generate a passphrase, which is usually easier to remember than a random string of gibberish.  Some of the “more experienced” among us will recognize this format as AOL-style passwords, which were printed on the 3.5″ floppies we received in the mail or our PC magazines of the day.

In 2014, we have nearly a decade of breaches and crappy passwords being stolen.  Even today, weak and obvious passwords are some of the most popular choices.  My hope is passphrases may become more of a standard, but I doubt it.  Some of the text of the original article is republished below; some of the links are broken, and I’ve replaced them where I could find a suitable alternative.

You can find the updated code in my BitBucket Git repo, at https://bitbucket.org/rjdudley/passphrase.  It’s pretty simple–one library project, a few tests in another project, and a console app to display the passphrases.  Use the library wherever it will run if you so wish, or fire up the console app anytime you need a good passphrase yourself.

Why Passphrases?

Perhaps first we should ask “What is a passphrase?”  Wikipedia may say it best:

A passphrase is a collection of ‘words’ used for access control, typically used to gain access to a computer system.

Passphrases were first proposed in 1981 by Sigmund Porter. Passphrases are distinguished from passwords by their virtue of being comprised of several words separated by spaces. Passphrases can satisfy even stringent security requirements, while being easier for the users to remember (http://technet.microsoft.com/library/cc512613.aspx). It’s this combination of complexity and ease of remembrance that make passphrases a good part of a password policy.

Our decision to use passphrases included another reason. By using passphrases when a user’s account is set up, we hoped to set an example to our users to use passphrases as well. We hoped that users would follow our example and choose passphrases they could remember easily, and that would be more than their dog’s name concatenated with a number 1. As a precedent, I cited that AOL has for years used multiple word passphrases as the login associated with all those floppies and CDs they send out. PGP and its variants also require using secure passphrases as your private key.

Recommended Passphrase Best Practices

With the intrinsic strength of some of the modern encryption, authentication, and message digest algorithms such as RSA, MD5, SHS and IDEA the user password or phrase is becoming more and more the focus of vulnerability. (http://www.totse.com/en/privacy/encryption/passch.html)

String passphrases are only one part of a comprehensive security policy. For additional security, you should include other best practices in your application’s login components. Microsoft makes a number of recommendations for Windows networks which are also applicable for ASP.NET applications (http://technet.microsoft.com/library/cc162924). These recommendations include:

  • Enforcing strong passwords
  • Ensure regular password changes
  • Maintain a history to prevent immediate reuse
  • Lock out accounts after a certain number of failed attempts

In a very good series of articles, Jesper Johansson reiterates many of these recommendations (http://technet.microsoft.com/library/cc512624.aspx), but disagrees about using account lockout policies. Several myths surrounding Windows passwords are addressed by Mark Burnett (http://online.securityfocus.com/infocus/1554/), and although focused on Windows passwords, some of the information is also applicable to ASP.NET applications. Designing a component that includes these recommendations is beyond the scope of this article, but you should familiarize yourself with these recommendations and incorporate the pertinent ones into your application.

Generating Passphrases

FAQ: How do I choose a good password or phrase?

ANS: Shocking nonsense makes the most sense. (http://virtualschool.edu/mon/Crypto/PGPPassPhraseFAQ.html)

There are a number of methods for generating passwords and passphrases. In this article, we’ll modify a method known as Diceware (http://world.std.com/~reinhold/diceware.html). This method consists of a numbered word list and five dice. Each word is assigned a 5-digit number, with only numbers 1-6 at each position, and covering every combination of numbers. The five dice are rolled, and the numbers are read from each face to form a 5-digit number. This number is cross-referenced with a word in the word list, which is then the first word in the passphrase. This process is repeated until the requisite length or number of words has been reached.

Instead of rolling dice, we’ll use pseudo-random number generators to simulate dice rolls. To make cross referencing easier, we’ll use the original wordlist loaded as a Dictionary object.  This version of the generator uses the RngCryptoServiceProvider to simulate the rolls of the dice.

A Simple Unit Test Script for SQL Server

As we develop applications, it often makes sense to put some functions in a SQL CLR assembly.  As these assemblies are deployed through test, beta and production environments we need an easy way to ensure the assemblies have been updated correctly and are functioning as designed.  Many things can go wrong—incorrect permissions, deploying the wrong version of an assembly, even assumptions we made in the business logic.  I’m a big fan of RedGate’s SQL Test plugin—it’s an attractive SSMS plugin which organizes and runs tests via an easy to read panel—and the tSQLt (a unit testing framework for SQL Serve) which SQL Test is based on.  My next post will show recreate the testing strategy shown here using SQL Test.

One downside to tSQt is that you have to install a number of stored procedures and functions into your database.  Depending on your choice of database, permissions in your database and willingness to add additional objects to your database which aren’t drectly related to the data, using tSQLt may not be an option.  Although we’re testing a user defined function, we could also test stored procedures and SQLCLR assemblies.  This technique is also not limited to SQL Server—I use this in both SQL Server and VistaDB.  As you build out your application, you may want to consider adding diagnostic pages (or screens) where you can execute these testing procedures should your application start throwing errors.

The function we’ll be testing will be simple—we’ll pass in an nvarchar, and return the same nvarchar with the word “wombat” added to the end.

Start by creating a custom function:

CREATE FUNCTION [AddWombat]
( @phrase NVarChar(50)
)
RETURNS NVarChar
AS
BEGIN
return @phrase + ' wombat'
END

Now it’s time for the test harness.  I’m using VistaDB, creating a stored procedure and using table variables, syntax supported by more recent versions of SQL Server.  You may need to optimize for the database you’re using.  The first table holds our test cases—the test name, the value we want to pass into our function, and the expected result.  We then loop through all the tests, calling the function we want to test, and updating the result.  Our result comparison is very simple, similar to an Assert.AreEqual.  With a little more work, additional test types could be added, and the test indicated in the test case.

create procedure TestAddWombat as
begin

DECLARE @testcases TABLE(TestNumber INT IDENTITY(1,1) PRIMARY KEY,
TestName NVARCHAR(100),
TestCase NVARCHAR(100),
Expected NVARCHAR(100),
Result NVARCHAR(20));

DECLARE @testname VARCHAR(100);
declare @testcase nvarchar(MAX);
DECLARE @expected NVARCHAR(MAX);
DECLARE @result NVARCHAR(max);

DECLARE @testCaseCount INT;
DECLARE @counter INT;

SET @counter = 1;

INSERT INTO @testcases
( TestName ,
TestCase ,
Expected
)

VALUES  ( N'should_return_hello_wombat' , -- TestName - nvarchar(100)
N'Hello, ' , -- TestCase - nvarchar(100)
N'Hello, wombat'  -- Expected - nvarchar(100)
)

INSERT INTO @testcases
( TestName ,
TestCase ,
Expected
)

VALUES  ( N'should_return_null' , -- TestName - nvarchar(100)
NULL , -- TestCase - nvarchar(100)
NULL -- Expected - nvarchar(100)
)

SELECT @testCaseCount = count(*) FROM @testcases

WHILE @counter <= @testCaseCount

BEGIN

SELECT @testname = TestName,
@expected = Expected,
@testcase = TestCase
FROM @testcases
WHERE TestNumber = @counter

SELECT @result = dbo.AddWombat(@testcase)

IF(@expected is NULL)
BEGIN
IF(@result is NULL)
UPDATE @testcases SET Result = 'True' WHERE TestNumber = @counter
ELSE
UPDATE @testcases SET Result = 'False' WHERE TestNumber = @counter
END
ELSE
BEGIN
IF(@expected = @result)
UPDATE @testcases SET Result = 'True' WHERE TestNumber = @counter
ELSE
UPDATE @testcases SET Result = 'False' WHERE TestNumber = @counter
END
SET @counter = @counter + 1;

END

SELECT * FROM @testcases

end

 

 

Running this sproc returns the following results:

 

image

 

From here, we have a very basic test harness which we can easily add additional test cases, and easily extend.  TDD is a best practice in any aspect of application development, but database development is often not treated as application development and is far behind adopting practices common to C# developers.  This script is a simple way to introduce some TDD into database development.

Hosting your own URL shortener with YOURLS, Azure Web Site and WebMatrix

For a couple years now, I’ve been paying for a private-label URL-shortening service.  I’ve amassed hundreds of links, and don’t want to lose any of the shortcuts, but the cost has gone up with no change in features.  I would love to have better metrics, and certainly a lower cost.  Having a small Azure benefit gives me a level of free hosting with an Azure website.  I hunted around and found several open source URL shorteners.  I decided on using YOURLS, a PHP-based application with all the features I want and then some (nice charts, plugins, social sharing, an API and bookmarklets).  Plus, it looked insanely easy to install.  Here’s how I set up YOURLS on Azure (this blog post took longer and is more difficult to read than the actual process, it was that easy).  There are a couple paths you can follow here, yours may differ from mine since I already had some existing services set up.

Step 1: MySQL on Azure

YOURLS uses MySQL as its database.  There are two primary ways you can host MySQL on Azure:

  • ClearDB offers hosted MySQL databases on Azure, with a 20 MB developer instance for free.  This is plenty to play around with, but if you want more you can.  There are two ways to set up CleaDB on Azure: How to Create a MySQL Database in Windows Azure.
  • Spin up a VM (Linux or Windows) and host a MySQL instance on it.  This is probably the more expensive option, but gives you the utmost control.  If you wanted to use this VM to host your YOURLS, you could, but that’s another blog post.

I already had my developer instance set up from an earlier WordPress experiment.  I created this instance as a linked resource for Windows Azure Web Site previously, which is an easy way to get started, but limits you to the 20 MB limit.

Either way, follow one of the sets of instructions at http://www.windowsazure.com/en-us/develop/php/common-tasks/create-mysql-database/ to get started.

If you’re sharing a MySQL database, by default the Yourls tables are created with yourls_ prefix, so you can separate new tables from existing ones.

Step 2: Creating the Web Site

As with the MySQL database, you can do this in one of two ways.  If you have already followed, or plan to follow, the instructions for creating a MySQL database as a linked resource, use that and skip this step.  In that process, you’ll create a site in the Azure portal, then create a CleaDB MySQL instance linked to the site.

If you have a existing MySQL database, but need to create the website, you can create the site from within WebMatrix 3, which is what I did (if you don’t have WebMatrix 3, you will need to update to the latest version).

From the start screen, New >> Empty Site starts the process.

image

The next step is to find a unique name for your site, and a location to host it.  At this step you’re configuring the long name for your site.  Later you can configure a custom URL for your website.

 

image

A local and remote site are created

image

image

 

At this point, you’re read to start working.  You’ll be working in a local version of your website, not directly in the website.

Step 3: Installing and Configuring YOURLS

To get started this way, download the most recent version from https://github.com/YOURLS/YOURLS/tags (cloning and publishing via Git will be discussed in another post).  Unzip the download into the folder for the website created above.

Configuration options are explained at http://yourls.org/#Config.  The main ones to configure are:

  1. YOURLS_DB_USER
  2. YOURLS_DB_PASS (remember this will be stored in plain text)
  3. YOURLS_DB_NAME
  4. YOURLS_DB_HOST
  5. YOURLS_DB_PREFIX (this is how you can separate YOURLS tables from any others in the same MySQL database)
  6. YOURLS_SITE (use the temporary URL until DNS propagates)
  7. YOURLS_COOKIEKEY (generate at http://yourls.org/cookie)
  8. $yourls_user_passwords (this is how you’ll log into the admin portal, you can encrypt these per https://github.com/YOURLS/YOURLS/wiki/Username-Passwords).

Do not put the config file in a publicly available location!  You have secrets in this file, make sure it stays private.

In order to view admin pages, you’ll also need to add a web.config to the root folder of your YOURLS site; see https://github.com/YOURLS/YOURLS/wiki/Web-Config-IIS for a sample file.

Step 4: Publishing to Azure and Installing Database Tables

If you used WebMatrix to create your Azure Web Site, all you need to do now is click Publish and your files will be transferred automatically.  If you created your database and site via the Azure Portal, you’ll be prompted to either choose an existing Azure Web Site, use WebDeploy, or manually configure FTP.  This is a one-time configuration—every subsequent time you can just hit Publish.

After the site is deployed, there is a one time installation.  Simply go to http://<yoursite>.cloudapp.net/admin, log in with the credentials you saved in the config file, and follow whatever prompts you’re given.

Step 5: Custom URL on Azure

Custom URLs for Azure Web Sites are not technically free, but you can apply Azure credits via an MSDN subscription (which I do) or pony up for a paid subscription.  Instructions for configuring a custom domain name for an Azure Web Site are at http://www.windowsazure.com/en-us/develop/net/common-tasks/custom-dns-web-site/.

You’ll need to have custom DNS for your site, usually this is done by using your registrar’s nameservers and custom DNS settings.  If your registrar doesn’t offer such a service, look into a service like https://dnsimple.com/.  You’ll need to configure both a CNAME and an A record.  Do not forward your domain name.  After you configure your DNS, it’ll take a day or so to propagate completely.  Once the DNS is propagated, you’ll need to edit the config file and set the YOURLS_SITE to the custom domain name and republish.

Step 6: Track those clicks!

You should be fully running, you can extend YOURLS with some of the plugins found at https://github.com/YOURLS/YOURLS/wiki/Plugin-List.

Happy shortening!

How to buy tickets for CodeMash

CodeMash is a very, very popular conference—demand for tickets far exceeds supply.  CodeMash sells out in minutes, and try as they might, the organizers simply can’t make everyone happy, nor assure anyone other than speakers and sponsors get tickets.  Whether you’re registering as a returning alum, or in the general admission, here are a few hints to make your registration easier.

Update: General tickets for CodeMash 2014 go on sale at 3:05 PM ET on 10/29.  If you didn’t know this until now, pay special attention to the section immediately below.  To check the time, you should probably use something accurate, like your cell phone, rather than your watch or the clock on the microwave.

Pay attention to the details

The organizers communicate a lot of information in the Google Group at https://groups.google.com/forum/m/?fromgroups#!forum/codemash.  Pay attention to the details—the day and time registration starts, when alumni emails are being sent.  If you don’t get your alumni email when you should, check your spam folder.  Contact the organizers in the Google Group within a day of them sending the emails.  Do not wait until the last minute!  Set appointments in whatever calendar app you use so you don’t miss the start times.

Tickets are purchased via EventBrite, not through the CodeMash site.  Don’t go to the CodeMash site to register—there is nothing there for you.

Also, understand the rules.  Alumni emails are sent only to attendees from the previous year.  If you went two years ago but not last year, no dice.

Anytime before registration starts

CodeMash uses EventBrite for tickets and PayPal for payments.  You can make your registration a little less stressful by creating accounts on each of these sites, and making sure your profile information is complete and accurate.  If you have a PayPal account but don’t want to use those funds to pay for your tickets, you may need to create a new account and connect it to the credit card you want to use.

Pro tip: use a personal email address to register!  People change jobs, and alumni emails are sent to addresses on record.  Use an address you won’t lose if your employment changes.

Day of alumni registration – reserve hotel room

The CodeMash hotel room block opens up at 8am ET on the day of Alumni registration.  You need to call and say you’re reserving a room for CodeMash.  Be patient, the Kalahari registration staff knows the drill and is awesome.

If you don’t get a room at the Kalahari, make a reservation at a nearby venue, and check back.  There is a great deal of room speculation, and as the speculators don’t get tickets, the rooms free up again.  Continue right up until the conference starts.  Also, in the Google Group, there are often requests for roommates and carpools.

About 10 minutes before registration

Open two tabs in your browser and log into Eventbrite and PayPal.  Confirm your information is correct.  After you log into Eventbrite, open the CodeMash registration page.

Decide what ticket you want—conference only (Thurs & Fri), conference plus one day of precompiler (Weds-Fri), or conference plus two days of precompiler (Tues-Fri).  Make up your mind before you try and register—going back during registration has caused problems and delayed or prevented people from registering.

If you buy a conference only and decide you want to upgrade to precompiler, you can sometimes upgrade at a later date, subject to availability.  No guarantees.

At registration time

Until the exact second registration starts, you’ll see a “no tickets available” (or similar) message.  Use control+F5 to reload the page over and over.  When you see a list of tickets, choose the one you want, and continue registration.  Now, relax.  The worst is over—you have 45 minutes to complete your purchase, and Eventbrite shows you a countdown clock.  Since you should be already logged in, you just need to confirm your information and continue on.

To pay, you’ll be transferred to PayPal.  You should still be logged in, but if not, log in again.  Again, be calm.  45 minutes.  Choose your payment method, complete your purchase.  Make sure you click the link to go back to Eventbrite and complete your purchase!  Once you’re transferred back to Eventbrite, you should see a purchase confirmation.  You’re done, celebrate all over social media and start planning your outfits.

If you don’t get a ticket

Ticket sessions are reserved for 45 minutes.  If a ticket isn’t purchased within 45 minutes, it’s released back into the pool.  About 40 minutes after registration started, come back and try again.  Keep trying, since sessions all started at different times.  If you still don’t get a ticket, put yourself on the wait list and keep watching the Google Group—tickets are canceled up until the conference starts, especially in the final week leading up to the conference and you may be able to score one of these.

If you get a ticket

Check out my post An Unofficial Guide to CodeMash: http://rjdudley.com/blog/2012/12/29/an-unofficial-guide-to-codemash/

If you can’t get a room at the Kalahari

There are a lot of nearby hotels in Sandusky, including a Great Wolf Lodge (that’ll show ’em–get admission to two indoor water parks!).  Grab a room at a nearby hotel and watch the Google group–people’s plans change and rooms become available, especially in the two weeks leading up to CodeMash.  The Kalahari will have a shuttle to the nearest hotels, see https://groups.google.com/forum/m/?fromgroups#!topic/codemash/sjL8fHEcPrE.

Review: Acer Iconia W3

I was an attendee at Build 2013, and received an Acer Iconia W3 and accessories as an attendee gift.  Now that I’ve had a couple months to use the device, it’s time to share my thoughts.  If you’ve ever said, “I’d love an iPad or a Kindle, but I wish I could use Office on them”, the W3 is definitely something to consider.

First Impressions

The Iconia W3 is designed to be held and work in portrait mode; others (such as the Surface) are designed to sit on a table in a widescreen format like a tiny laptop.  When you pick it up, you can tell the W3 is meant to be held in your hand and used for reading and consuming media.  Pre-installed Kindle, Hulu Plus and Netflix apps confirm this idea.  At a tad over one pound, the Iconia W3 feels solid but not too heavy.  The screen is bright and easy to read, and the new Windows 8 start screen makes navigating applications easy.  The screen is 8.1”, and fits nicely in one hand for me.  The advantage to a full Windows tablet is that actual Windows applications run on it—most notably Office, including Outlook.  Complex spreadsheets would be difficult on the small screen, but basic word processing and email are fine.  The Windows Store is packed full of apps, more than likely everything you want is now available.  You can browse the Windows app store at http://windows.microsoft.com/en-us/windows-8/apps.

Comparable Tablets

To put the Iconia W3 into the proper context, it’s similar to the iPad Mini, Galaxy Note 7 and the Kindle Fire HD.  Each of these tablets has a screen around 7”-8” and other similar features.  I do not have any of these devices, so I can’t do a side-by-side comparison.  If you’re looking for a smaller tablet device, this is your comparison group.  The Iconia is cheaper than an iPad Mini, search your favorite purchasing sites for prices on the other devices.

Specifications

The Iconia W3 is comparable in specs to some of the more recent netbook computers:

  • 32 bit Windows 8 (current Win 8 versions can be upgraded to 8.1 for free)
  • 2 GB SDRAM (RAM cannot be upgraded since the device is sealed)
  • 32 or 64 GB Flash Storage (the 64 GB version has 49 GB of actual usable space due to the OS and pre-installed apps, but storage can be increased with a Micro SD card)
  • 8.1” screen with 5-point touch control (high end devices have 10-point touch, so the W3 supports the basic gestures but not some of the more advanced ones)
  • dual core 1.5 GHz Atom Z2760 processor, which is hyperthreaded.  Don’t let the seemingly slow speed of the processor fool you, this is pretty good.  If you’re used to seeing desktops at around 3 GHz, this may seem weird, but GHz is really an old benchmark that no longer accurately describes a processor’s performance.  It’s just the thing everyone knows.

Pros

  • Small size
  • Windows 8
  • Micro USB 2.0 (you need a USB 2.0 dongle for a full size USB port)
  • Micro HDMI port (to connect to a TV or a monitor, you need to get a micro HDMI to HDMI cable, or a micro HDMI dongle to use a regular HDMI cable)
  • Mini SD card slot, so you can add up to 32GB more storage
  • Long battery life
  • 1280 x 800 px 8.1” screen, capable of displaying 720 px HD
  • Bluetooth, so you can connect a mouse or keyboard

Unlike iOS, Windows 8 allows you to set up a number of users, and can designate child accounts.  With Windows 8 Family Safety, you can set limits of web browsing, game play and application usage, and receive reports via email or online of your child’s activity.  The picture password makes it easy for even small kids to log in, and I took a few minutes to arrange my daughter’s start screen with her stuff.

Acer touts an 8 hour battery life for the W3, and I routinely get that long from a full charge.  This is great for long trips.  Because of the small size, it’s very easy to carry around with you and use in a car, on an airplane.

The USB port allows you to connect printers, external DVD drives, thumb drives and other accessories directly to it (which is not something you can do with any other tablet).  You can also run iTunes on the W3 and manage an iPad/iPod/iPhone (take that Apple!).  Chances are all your cords are regular sized USB, so you’ll need a dongle for a regular USB connection.  You can connect a multi-USB thingy to your dongle, and then attach a bunch of devices.  For something really cool, look at the Plugable USB 2.0 Docking Station, which is great if you want to connect a monitor and several peripherals in a desktop scenario (great for students!) while still having a really portable tablet.

Having a daughter, we’ve purchased many DVDs, which I’ve ripped to video files (yes, I buy and keep the DVDs so it’s all legal as I understand it to be).  When we travel, I load up the videos onto the W3, and I pack an HDMI cable.  The video app is very easy for a child to use, so she can consume her media on the go, and in the hotel we can hook up to the TV via the HDMI cable.

Besides iTunes, you can also add music and videos from Amazon Prime or Microsoft’s store, plus Hulu and Netflix.  The Zune Music Pass is now the XBox Music Pass, so you can have unlimited streaming with a subscription ($99 if you buy the full year).  There are many media options for the W3.

You can control your Xbox from the W3 using XBox SmartGlass, including the DVD controls, so if you have an XBox as an entertainment device this is great.  Also, you can connect to your XBox Live account and play a lot of connected games.

Cons

  • Weak WiFi radio
  • bad camera – 2MP front, 2MP rear
  • low screen quality – not 1020 HD quality screen, not as sharp as a retina display
  • Several reports of fellow attendees of DOA devices, or failures later.
  • Not the most precise touch—with larger fingers, should look into a touch stylus.

Some of my fellow attendees reported devices which were DOA, and Acer replaced them at the conference.  The WiFi radio is weaker than my iPad, but when I’m at home or a coffee shop that isn’t an issue.  That has been an issue in a hotel where the signal was wimpy also, but all my devices suffered there.

The W3 sports two cameras—one front, one back—but both are a low 2MP resolution.  By contrast, most smartphones are 5-8MP, so this is rather sad.  Good enough for Skype, but I wouldn’t try and preserve any precious memories with it.  The screen is not as sharp as a Retina display, and won’t display 1020 HD video without downscaling, so you lose quality there (720 HD is fine).

They’ve jammed a lot of pixels into a small screen, so it can be tricky to touch exactly what you want.  Many apps support pinch-zoom, which helps, but I also have a touch stylus (the ones with the rubber tip, same as an iPad) which I use when the zoom isn’t supported.

Accessories

Although you can use any Bluetooth keyboard with the Iconia, the Acer Bluetooth Keyboard is designed specifically for the Iconia W3.  It’s an almost full size keyboard which acts as a tablet stand.  When not in use, the tablet snaps into an indention in the bottom of the keyboard for easier transport.  It’s actually a really nice keyboard, and makes email and word processing much easier.  In fact, the bulk of this blog post was written on my W3.

Any Bluetooth mouse will work, and you can use the USB port for a wireless or wired mouse (again, dongle).  Bluetooth printers, speakers, etc  should all work.

We did receive a micro USB to USB dongle with the attendee kit, and you’ll definitely want to add one.  Like I mentioned above, I also pack an HDMI cable for use in a hotel room.

Can is really play Halo?

Yes, you can play certain expansions.  Here’s Halo: Spartan Assault in the Windows Store: http://apps.microsoft.com/windows/en-us/app/halo-spartan-assault/8fe2d694-baa2-4011-99c0-3a22216223bb.

Are there good apps for kids?

Plenty.  Fresh Paint is an amazing drawing/painting app (it was in the commercial with the really big tablet and little girl), and it’s free.  My daughter really enjoys the Disney Fairies Hidden Treasures, but be careful if you connect the W3 to your XBox live account—all my friends playing Call of Duty see me level up in Disney Fairies.  Angry Birds Star Wars and Angry Birds Space are also available.  It’s worth following the link above and seeing if something you like is available, or find something you didn’t know about.

Summary

Even though the Iconia W3 is a full PC, given the form factor it’s probably best to think of this as a powerful media device which can run Office rather than as a full PC.  To me, the W3 is a very decent secondary device.  For the cost of a keyboard and monitor, you’re in the range of a capable laptop.  For someone who wants both extreme portability and a larger screen, but only one device, a W3 and an external monitor could be a good compromise.  Having said that, I’m a software engineer and my tech requirements are a little higher than average—someone who only does email, Facebook, some light Word or Excel and loves to read, this is a great device to consider.

Developer Productivity Tip: The Personal Standup

This is one of my favorite productivity practices, and one I’ve found to help me maintain a high level of productivity. Every morning, prior to my team’s standup, I had my own personal standup for 10-15 minutes. The time was blocked off in my calendar as a standing appointment so no one could schedule over my time.

I’d use this time to reflect on the previous day and plan the current day, and go into team standup with a complete update and plan. This is a technique I picked up from my dad, who used a DayTimer well into retirement. I kept a daily diary of ideas and what I worked on, usually using a pocket-sized Moleskine Classic. My personal preference is plain paper, but whatever fits your style is perfect. Later on I started using an Emergent Task Planner every day. Whatever you use, it shouldn’t be a clumsy burden–it should fit your workflow and style. You should actually want to plan and record (get a good pen, too). This is something I continue to do today.

2013-04-25T18-53-07_0 (Medium)

Every morning I looked back at the tasks which needed continued, what I had completed, any blocks, and set up a plan for the day. From then it was smooth sailing through the day. Most of the time.