My Software Notes

Useful things I discover

Archive for June 2009

Adding that “all” element to a combo box

leave a comment »

In the past when data binding a combo box, I’ve found that adding the “(all)” or “(none)” element to the top of the list has always seems unaesthetic to me.  A funny choice of words, you may think, but to me well written code is aesthetic and adding that top element just seemed messy. 

If you were using a data set then you had to create a row and stick it in the table in the data set and then sort the thing so your element would come to the top and then bind to the table or to the data view.

If you weren’t data binding but were loading elements in a loop, you still had to stick that “all” element in there to make sure it got on top.

Binding to a collection of objects was a bit cleaner but I still didn’t much care for it.

But now we have LINQ and at last I’ve found the aesthetic way to do it!

using (var db = new ProductDataContext())
{
    var results = from p in db.ProductTypes
                  orderby p.Description
                  select p;
    prodTypeCollection.AddRange(results.ToList());
}
//Add an "All" item to the top of the list
var all = new ProductType()
{
    ProductTypeCode = PRODUCT_TYPE_CODE_ALL,
    Description = PRODUCT_TYPE_DESCR_ALL
};
prodTypeCollection.Insert(0, all);

I like LINQ.  It is making my job a hell-uva-lot easier and more enjoyable.

Written by gsdwriter

June 7, 2009 at 8:56 pm

Posted in Languages, LINQ

Tagged with , ,

Fun with lambda expressions

with one comment

While creating C# 3.0 equivalents to the Martin Fowler example Closures in Ruby in my last post (Closures in C# (better known as lambdas)), I was having so much fun that I carried on and came up with this:

Here are the “inline” versions (the PaidMore example is my favorite – just look at the original C# 2.0 version and you’ll see how far we have come):

Example Inline C# 3.0
Managers
var managers = emps.FindAll(e => e.IsManager);
//OR
var managers = emps.Where(e => e.IsManager);
HighPaid
var highPaidEmps = emps.FindAll(e => e.Salary > threshold);
//OR
var highPaidEmps = emps.Where(e => e.Salary > threshold);
PaidMore
Func<int, Predicate<Employee>> PaidMore = amount => e => e.Salary > amount;
Using PaidMore
var highPaid = PaidMore(150);

var areHighPaid = emps.FindAll(highPaid);
//OR
var areHighPaid = emps.Where(e => highPaid(e));

(Note that “Where()” returns an anonymous type so if you wanted a List<Employee> you’d need to call the “ToList()” method.)

I also did LINQ expression versions of all the above, and whilst they are implementing lambdas in the background, you can’t see them in the code, so I didn’t include them here.

Getting a little off the subject of closures, another cool use of the C# 3.0 features is to implement PaidMore as an extension method:

public static bool PaidMore(this Employee e, int amount)
{
    return e.Salary > amount;
}

Then call it like this:

var areHighPaid = emps.Where(e => e.PaidMore(150));

Or

var areHighPaid = from e in emps
		where e.PaidMore(150)
		select e;

Closures/Lambdas not only open up a lot of possibilities but also add a new element of fun to C# programming.

Written by gsdwriter

June 7, 2009 at 3:06 pm

Posted in Languages

Tagged with , ,

Closures in C# (better known as lambdas)

with 2 comments

I’m a big fan of Martin Fowler, the author of such brilliant and practical books as Refactoring: Improving the Design of Existing Code, Patterns of Enterprise Application Architecture and UML Distilled (these are books that every developer should have, study and refer to often). Today I was reading an article on his web site and I came across the term “Closure“. I’d come across it some years ago whilst reading another article on his site and I’d gotten a basic concept of what it meant at the time, but I never really used it, so the concept became foggy.

Today when looking over his description of what a closure is I realized the obvious: for us .NET types a closure can be accomplished with a lambda expression.

I read the full article on Martin’s site then clicked on the “C#” link at the bottom and was taken to the old blog of Joe Walnes, where he gives C# 2.0 versions of Martin’s closure examples.

I looked at them and thought how would you do these in C# 3.0? Well, I couldn’t resist the challenge, so here they are.

The main difference now between the Ruby examples and the C# examples is the type declarations in C#: you have to say what the methods return, what types their parameters are and that threshold is an int. But, notice that with the lambda expressions you don’t have to specify the type because it is inferred by the compiler.

While I was doing this I got a bit carried away and created more ways of implementing these, but to keep the emphasis on the comparison between the original Ruby closures and the .NET lambdas I will put the other versions in another post: Fun with lambda expressions.

  Ruby
1.
def managers(emps)
  return emps.select {|e| e.isManager}
end
2.
def highPaid(emps)
  threshold = 150
  return emps.select {|e| e.salary > threshold}
end
3.
def paidMore(amount)
  return Proc.new {|e| e.salary > amount}
end
4.
highPaid = paidMore(150)
john = Employee.new
john.salary = 200
print highPaid.call(john)

 

  C# 3.0
1.
public List<Employee> Managers(List<Employee> emps) {
    return emps.FindAll(e => e.IsManager);
}
2.
public List<Employee> HighPaid(List<Employee> emps) {
    int threshold = 150;
    return emps.FindAll(e => e.Salary > threshold);
}
3.
public Predicate<Employee> PaidMore(int amount) {
    return e => e.Salary > amount;
}
4.
var highPaid = PaidMore(150);
var john = new Employee();
john.Salary = 200;
Console.WriteLine(highPaid(john));

Written by gsdwriter

June 6, 2009 at 2:04 pm

Posted in Languages

Tagged with , ,

Sharding Notes: First rule of database sharding

leave a comment »

As far as I’m concerned, the first rule of database sharding is “You must have a data access layer“.

If you have architected your system in a “client-server” style were data access code is embedded in the client code then the effort required to shard is going to be huge, partially because you have so many places in your code to address and partially because of the vast amount of testing that is going to be required because you have likely touched every source file in your system.

I hit this problem some time ago.  It wasn’t on a sharding project but something similar.  The project was writing the next version of a system that had been written in VB6 by some inexperienced developers.  Data access code and dynamically built SQL statements were all over the place.  Their idea of a data access layer was a “Db” module that had methods for running queries and returning DAO result sets.  E.g.,

Set rs = Db.Execute("Select * from Customers where " _
    + strCol + " = " + strCustName)

And about 20% of the time they would directly call DAO and not user their own “DAL”. Yuck!

We began moving the functionality over to .NET piece by piece.  I created a three-layer style architecture for the .NET pieces: UI Layer, Domain Layer (sometimes called Business Logic Layer) and Data Access Layer.  We used COM Interop to call the .NET code from the VB6 exe.  (I wish I could have created a .NET exe and converted the VB6 part to a DLL but … well that’s a subject for another post.)

After some successful releases the client decided they wanted to move over to using SQL Server instead of MS Access.  We got a SQL Server version of the database created relatively quickly and then we moved on to converting the code to use the new database.

To convert the .NET code took one developer 3 days and that included extensive unit testing.  This was because all data access code was in one place and the Domain and UI Layers knew nothing about the database: not its structure, not its constraints, not its table names, not its column names, nothing.  It would have taken less time if it were not for the fact that we had to tweek some of the Access SQL so it would play nice with SQL Server.

To convert the VB6 code took three developers over two months to get about 75% done (without all the needed testing) and was then put on hold because there was a deadline to get out some high priority new features to the system users. 

Why so long?  It wasn’t the developers, these were smart, hardworking  guys.  No, it was simply the lack of a data access layer.

The same applies to sharding a database.  If you don’t have your data access code isolated in one area then I’d suggest you do that first before you even think of sharding. 

It is probably possible to shard without a data access layer, but then it’s probably possible to stab yourself with a machete and not bleed to death, but do you really want to risk it?

Written by gsdwriter

June 4, 2009 at 11:22 am

Posted in Database Sharding

Tagged with ,

Replacing Constructors with Object Initializers

leave a comment »

I was refactoring a class (in C#)  that had gotten big and messy and I realized that several of the static “factory-ish” methods that returned an instance of the class type were really not needed.  I call them “factory-ish” because they looked like they were doing the job you’d expect of a factory method (return a specific instance of a parent type based on a selector value) but what was really going on inside was simply passing a lookup value to a constructor and returning the object.  There was no hierarchy just instances of the same class.  The factory-ish methods all looked like this:

public static Thing FindByLookupValueA(string lookupA)
{
    var thing = new Thing(lookupA);
    thing.Load();
    return thing;
}

There were five methods like this.  They had grown over the years one by one, like warts on your neck just below the collar so you don’t notice them.  There were also constructors matching these static methods.  It just seemed unnecessary.

So I began replacing the static methods with calls to constructors.  At first I used the existing constructors like “Thing(int lookup)”, “Thing(string lookup)”, etc.  Then I thought, “wait a minute!”.  All those constructors did was:

public Thing(string lookupA) : this()
{
    LookupA = lookupA:
}

What a waste of typing.

If all a constructor does is set values on properties and nothing else – no calculations, no special initialization, no nothing – then it doesn’t need to exist.  Use an object initializer instead:

var thing = new Thing() {  LookupA = lookupA  };

Simple as that.

Doing it this way I got rid of five static methods and three constructors.  That’s eight unnecessary public API members of the class gone.  Simplicity rules!

KISS!

Written by gsdwriter

June 3, 2009 at 1:15 pm

Posted in Languages

Tagged with ,

Sharding Notes – Splitting Tables

with one comment

We hit some tables where some records applied to only one shard but others applied to multiple shards.

As an example, one of the tables was Users.  The database is sharded on the Companies table.  Many users belong to only one company (e.g. employees of a company) but there were some (such as in-house support people and super-admins) who belong to multiple companies.

The possible solutions we came up with were:

  1. Split the tables according to the different records (Table per Type or Table per Concrete Class).  For example, keep the Users table for people who belong to only one company and create an AdminUsers table for those who belong to multiple companies.
  2. Add a discriminator column (e.g., UserType with values of ‘Regular’ or ‘Admin’) and handle the records according to the type (Table per Hierarchy).
  3. Move the split table that applies to multiple shards into a “master” database so it is not on the shards and query it there.  E.g., we split Users into Users and AdminUsers and move AdminUsers onto the master database. 
  4. When loading a user do a cross-shard query to get all the companies/shards the entity is on.  Then do all further CRUD operations on the entity by hitting the specific shards you found in the first query.  E.g., User #5 administers company #7 on shard 1, company #10 on shard 2 and company #15 on shard 3.  He does not administer any companies on shards 4 or 5.  Further CRUD for that user will hit shards 1, 2 and 3.

My preference is #1, but I think we’ll be going with #4 on all the “special” tables because the amount of extra work in implementing #1 right now is way more than we have the time or  resources for.  Perhaps once we have the sharded system up and running we may switch to #1, but that is in the future and a lot depends on the performance for #4.  Luckily most of the “special” tables (other than User) are not high traffic, so we’re hoping it won’t be a problem.

Of course your own situation will probably be different so you should evaluate the pros and cons of these methods for your particular situation.   And you may come up with other ways to do it.  If you do, please leave a comment describing your solution.

Good Sharding Article

Here is a good intro article on sharding: Database Sharding.

Written by gsdwriter

June 1, 2009 at 2:36 pm

Posted in Database Sharding

Tagged with ,