It felt like the cirque de soleil of conferences with a healthy mix of audience engagement, ranging from an abrupt ovation for our country with a national anthem who intent was to rebase the audience back to our country’s patriotic roots, to motivational talks which engaged the audience in a 30 second breathing exercise and acknowledgement of people in adjacent seats–all designed to underline the prominence of “presence”, “awareness” and connecting. The topic on energy for a change focused on solutions and the Thar coal mines, whose 4% reserves are enough to sustain the entire country. Micro-finance seems to be another recurring theme in the financial and charitable circles and there was plenty of quantification and empirical evidence of it being implemented and working in Pakistan. All talks came from people who are out there on the field, accomplishing these things.

The talks had plenty of substance and mind fodder. It’s easy to get pigeon-holed into our respective trades so it was a good change to get a richer, broader perspective from people across the various walks of life.

It’s a shame that it will be another year before the next TEDx, but I’m hoping there are other forums in between that continue the discussion.

Building large scale systems like The Express Tribune news portal and turning it around in 6 months requires a team of determined masochists and some really quick thinking. One has to think about scalability, performance, security, architecture and pliability of the product. One also has to think about usability, information architecture and layouts. About user interactivity, community engagement and publication workflows. Integration with television and print and a coherent new media strategy.

But it’s here and it’s far from over. Going in, I was confident that we would easily come out ahead of the competition (whom I have written about here and here) but I never thought I would have so much fun doing it.

The Express Tribune website uses some existing platforms and our developer toolkit is quite powerful, but to get to where we wanted to go, we got neck deep in every aspect of the system.

Beyond being just a pretty website, it has some behind the scenes features where it really shines out. This is where the competition has a lot of catching up to do. For example, assigning headlines and stories to sections or updating the page layout happens directly from the section itself, rather than going in to some specialized administrative screen. The image management and carousels are first class features and not just an after thought. The News in Pictures and slide shows can be done in under 5 minutes. The pages load blazingly fast, because we optimized not only the caching mechanisms but also the web server, database server, application server and even the operating system.

The user interface follows some hard principles. For example, the comment preview feature is painstakingly simple and dynamic so users know exactly how their words will appear. The submit button is below the comment preview by design, so the user is forced to preview on his way to the submit button. The design follows a horizontal rhythm using grid-based layouts. The weather widget updates the weather without having to refresh the page and on the back end we do some very specific caching so we can handle thousands of users, yet provide the latest weather updates or auto updating stock charts. The alerts ticker is directly linked to the Express 24/7 television station which requires some trickery on the part of both, the television platform as well as the web platform.

The interesting bits about strategy are something I cannot talk about other than to say that if the website works well for you and you find yourself interacting more and more, it’s because we put the user first. And in the short and long term both, our strategy will allow us to surpass the competition and out-pace them so we maintain the lead.

With internet penetration growing the way it is, I have no doubt that The Express Tribune portal will provide common ground for a lot of avid readers and have interesting side effects.

The website is a big hoarding with no less than 20 advertisements on the front page while at the same time carries less than 100 words of actual news. And it looks really ugly.

This is what happens when you just don’t care about the readers. This is what happens when designers develop, developers design, business units dictate the roadmap and talent in general lacks. This is also what happens when you just don’t understand interaction design, information architecture, usability or have been oblivious to the paradigm shifts in online advertising and technological trends. This screen capture of the Jang News website clearly highlights all this.

The actual news content is highlighted in green boxes while the rest of the page is mostly advertisements. All ad slots highlighted in red are available for sale while unsold ad space is used for in-house products. The advertising tariffs section of the website indicates the type of ads available for sale on Jang News Online which includes video and expandable ads at a premium. The annoying expandable ads block access to the news until readers close them (shown in the screen capture) while video ads block the rest of the page because they take so long to load (Pakistan has very low broadband penetration in any case).

Jang News Online gets a considerable amount of traffic because the Jang News Group has entrenched itself as the face of news over the past 70 years that it has been around–a time during which it enjoyed little or no competition which helps explain their complacency.

Now, I am not even sure why an advertiser would want to publish one ad amongst 20 when it’s well known that users develop banner blindness and moreover the website’s ad space is extremely diluted and the screen interface is so cluttered.

The page reminds me of the days popup ads ran rampant until readers expressed rage and all major browsers (Internet Explorer, Firefox, Safari, et al) reacted by featuring popup blockers to put an end to it all. Unfortunately, in Jang’s case it’s not easy to block the spam. The website’s front page is practically an online hoarding which seems to take it’s cue from the million dollar homepage (whose sole purpose is to show advertisements) rather than a news site. One may easily pass this off as spam in its current state.

There are some valuable lessons in this for all budding web developers and web entrepreneurs.

Download PowerPoint: Online Strategy and Development in a Nutshell.

In a seemingly abrupt move, tr.im recently announced the decision to shut down at the end of this year blaming the bit.ly and Twitter partnership for driving them out of business. I have not used either service other than for test driving so I don’t have a bias either way but it makes sense for Twitter to go with best of the breed so it is unclear why tr.im needs to place the blame with either bit.ly or Twitter. The situation was pareto efficient and one or the other had to lose out. It makes sense for Twitter to partner with or acquire such a service because it is complimentary to their business (tiny posts with tiny URLs).

Soon after announcing their intent to go out of business, tr.im reversed their position and Eric Woodward offered to bear the costs of operations out of his own pocket if donations fell short–a move that is even worse than going out of business.

Revenue Model

The revenue model suggests that for the most part, such a service is very difficult to sustain. The revenue model for URL shortening services would depend either on showing disruptive advertisements or tracking user patterns and selling intelligence.

In the former case, users would probably stop using the service if they saw an ad before being redirect to long URL. In the latter case, tr.im would need considerable market share to track behavioural data.

Tr.im shot itself in the foot by claiming it will simply go belly up because the business is bad. There goes their chance of getting acquired for a good price.

All this is offset by the fact that it allowed tr.im to receive a lot of media attention and might drive up usage but smart users will shy away from a service that’s on life support and business continuity is of utmost importance. The smarter strategy was to get acquired much like bit.ly did.

Why go open?

The technical challenge of converting long URLs to short ones is trivial so the value-proposition to the community is not that great but let’s assume that since it’s free it’s good. The problem then again is that it will encourage users to create their own competing services to bit.ly and since the revenue model for the most part does not have a compelling monetization strategy, these services will eventually go out of business and intensify the link-rot problem.

The primary motivation for going open seems to be the move toward donations where the community would donate code as well as money to keep the servers running for the millions of links that will accumulate over time. This works well for Wikipedia because it is a natural monopoly and can fall back on ads if needed. And again, Wikipedia has completely different dynamics compared to tr.im which functions simply to shorten the length of links with equal or better services already available that pose a lesser risk of going bust.

A Nasty Time Bomb

When Eric Woodward proposes that he will fund this from his own pocket if donations fall short, the community should be concerned. What is something happens to Eric? Does he leave behind a trust fund? Not only that but the costs are ever-increasing because all links will need to be supported for all of time or thousands if not millions of links will break.

The longer the service stays in operation, the greater the risk it poses. If it goes out of business 2 years from now it might end taking out a couple million URLs with it which will be even worse than if it went out of business today.

So the costs will be ever-increasing, the risks will also be ever-increasing and there is no good revenue model to sustain the service. In fact if it becomes community owned, all data will be free so tr.im cannot depend on behavioural tracking data as a revenue strategy. Tr.im will continue to be a burden on the community and not only that, but if other unsustainable services come up replicating its code, the link rot problem will get compounded. And all those people who make donations will see their money go to waste when they realize the tr.im will need piles of cash, month after month with no other source of cash flow.

I hope I am wrong about this.

Microsoft’s (MSFT) fourth-quarter net income dropped 29% to $3.05 billion and revenue fell 17% to $13.10 billion. It was hurt by a global slump in demand for PCs and servers. Sales in the client division which produces Window OS, fell 29% and earnings fell 33%. Google will be introducing the Chrome OS soon while Microsoft will be launching it’s own Windows 7 in October of this year.

Apple (AAPL) reported third-quarter profits increase by 15% to $1.23 billion and revenue rose 12% to $8.33 billion. Apple sold 5.2 million iPhones, a 700% increase YoY. Macintosh saw 4% rise in shipments to 2.6 million units, however, revenue fell 8% to 3.33 billion due to price cuts on some MacBooks. iPod sales dropped 7% YoY to 10.2 million and revenues declined 11% to 1.49 billion. This is the natural effect of cannibalization by iPhone and iTouch.

Amazon (AMZN) said its second-quarter net slid 10% to $142 million, though sales increased 14% to $4.65 billion. The company, which announced an agreement to buy online retailer Zappos.com yesterday for $850 million, said it expects net sales of between $4.75 billion and $5.25 billion in the third quarter.

Yahoo (YHOO) said its second-quarter revenue slid 13% to $1.57 billion but profits rose 8% to $141.4 million due to layoffs of 700 employees to 13,000 total, and other cost-cutting measures. A Yahoo/Microsoft deal is back in the negotiating room as the companies seek to form a partnership against rival Google, which means the deal would have to be approved by regulators due to antitrust concerns. The companies have skirted the negotiations over the past 2 years until they came to head a few months ago with CEO Jerry Yang when negotiations failed and Yahoo stock fell considerably as the recession hit. Yahoo, this past Tuesday launched a new home page and announced a deal with AT&T to sell display advertising.

AMD (AMD) posted second quarter losses of $330 million while revenue was down from $1.36 billion to $1.18 billion. Margins for its chips business were down from 35% to 27% a stark contrast from rival Intel’s report of increasing gross margins. Intel’s revenues also in turn increased 12%. AMD blamed the declines on under-utilized factories which lead to a reduction in margins and also lower chip prices.

Google (GOOG) reported a revenue increase for 2.9% to $5.52 billion while profits rose 19% to $1.48 billion due to cost cuts and laying off some 300 employees. Google commands one-third of the $24.5 billion U.S. Internet advertising market. Mr. Schmidt reported a recovery in advertising spend in travel and shopping but finance sector remains weak. The bulk of the growth came for search advertisements as advertisers turn to more predictable and targetted ads on Google.

TomTom (TOM2), a dutch navigation equipment manufacturer reports 61% drop in second quarter profits to $28 million while the top line saw a decline of  19% to $522 million. The company expects to a total industry demand of 15 million Portable Navigation Device units in Europe and 17 million in North America for the complete fiscal year of which it expects that 11 to 12 million units will be of TomTom. The company also suffers stiff competition from the likes of iPhone and other similar phones in the market which provide a navigation facility among other things.

Wipro (WIT) reported a 12% rise in first-quarter profits to $210 million making it the third Indian software company to beat forecasts. Earlier, bigger rival Tata reported a 22% increase in profits while Infosys reported a 17% increase in profits. Wipro, India’s third largest software maker booked revenues from IT services, products, customer care and lighting businesses.  IT Services registered revenues of $1.03 billion which makes up 77% of Wipro’s revenue stream and margins in the same division grew from 20.9% to 22.3%.

LG Electronics, South Korea’s second largest consumer electronic company after Samsung, reported quarterly net income increase for 62% to $918 million and earnings increased 14% to 1.2 billion. LG expects strong outlook in LCD and handset business. The company sold nearly 30 million handsets in the quarter and operating margin on these handsets increased from 6.7% to 11% QoQ.

Elsewhere, Cisco (CSCO) signed a deal with Marriot hotels for telepresence services in 25 locations. Telepresence helps reduce travel costs and Mariott cancelled previous deals with HP (HPQ) telepresence solutions for $120K each plus $10K monthly fee. Last year alone, HP sold Cisco gear worth $1 billion but tough economic times has lead to tough competition between the two. Cisco’s top of the line telepresence sells for $300K vs HP’s $350K. BBC has dumped project Kangaroo, an Internet TV service that was planned in collaboration with two other television networks. The project was ruled out in February due to antitrust issues from the Competition Commission. BBC wrote off some $15 million in the project for the FY ended April 30. The BBC iPlayer gets 41 million+ hits a month and BBC is currently exploring other options for profitable online television broadcasting.

JavaScriptSerializer

The JavaScriptSerializer lives in the System.Web.Script.Serialization namespace and the usage is fairly straight forward for serialization:

JavaScriptSerializer serializer = new JavaScriptSerializer();
String json = serializer.Serialize(data);

For deserialization however, there is a minor annoyance in that the deserializer accepts a generic type along with the content:

serializer.Deserialize<T>(String s)

which can be a problem if the type T is not known at compile time and needs to be dynamic. The work around is a bit ugly as I learnt because it uses reflection to create a generic method but it works:

var result = typeof(JavaScriptSerializer).GetMethod("Deserialize")
             .MakeGenericMethod(JsonDataType)
             .Invoke(serializer, new object[] { inputContent });

The JavaScriptSerializer is excellent for general purpose serialization and deserialization, however, one downside is that it cannot handle circular references.

A useful feature of the JavaScriptSerializer is that you can also implement a custom JavaScriptConverter and pass that in to JavaScriptSerializer for fine-grained control over the serialization/deserialization. However, for it to be really useful you need to know the types at compile time and have references to those types. This really limits the usefulness of this feature because by referencing those classes your code becomes tightly coupled so you cannot easily use it in something like an MVC filter.

The JavaScriptSerializer includes all public fields and properties for serialization by default which makes it useful when working with auto-generated classes or if you don’t have access to the source. This is different from how DataContractJsonSerializer works.

DataContractJsonSerializer

The DataContractJsonSerializer lives in the System.Runtime.Serialization.Json namespace. Unlike the JavaScriptSerializer, the DataContractJsonSerializer is a contract-based serializer so classes or members need to have the [DataContract], [DataMember] or the [Serializable] attribute.

[Serializable]
class Person {
    public String Name;
    public Int32 Age;
}

...

String json;
using (var stream = new MemoryStream()) {
    DataContractJsonSerializer serializer =
       new DataContractJsonSerializer(data.GetType());
    serializer.WriteObject(stream, data);
    json = Encoding.UTF8.GetString(stream.ToArray());
}

And for deserialization:

using(var stream = new MemoryStream(Encoding.Unicode.GetBytes(json)) {
    Object data = serializer.ReadObject(stream);
}

Since DataContractJsonSerializer is contract based and relies on attributes, it is not particularly suitable for auto-generated classes (unless they have these attributes) or when source is not accessible because the relevant attributes cannot be added at runtime as far as I know. However, for other situations attributes are an easy way to control the serialization aspects.

The most compelling feature of the DataContractJsonSerializer is that it can handle complex object graphs with circular references. This makes it particularly useful with the Entity Framework 4.0 which now support T4 templates allowing required [Serializable] or [DataMember] attributes to be added to auto-generated classes.

Update: In .NET 3.5 SP1, DataContractSerializer added support for classes without attributes (or POCO objects).

Json in MVC

The System.Web.Mvc namespace contains the Json function intended for use within ASP.NET MVC. There is no real reason to use it outside of ASP.NET MVC when JavaScriptSerializer and DataContractJsonSerializer are available. This function makes it extremely easy to serialize objects to a JSON string. It is not intended for deserialization, however, serialization is just one call:

String content = Json(data);

POCO Support

If you are not up to date on POCO, quickly read up on POCO and Persistence Ignorance first. Entity Framework 4.0 has full support for POCO or Plain Old CLR Objects.

If you have a class called Person with name, address and phone number then that’s how it will stay. It will not have EntityKey or some other such properties. It does not need to implement any interface specific to Entity Framework nor does it need to add any special attributes on properties and methods.

You can pass this plain object around much more cleanly and your domain logic is not polluted with any persistence code. This is also useful for Test Driven Development (TDD)  or migrating out of EF to another framework down the line.

Entity Model Designer Improvements

Previously you had to design the database schema in SQL Server and then import that into the Entity Designer. With 4.0 you can build the model first in the Entity Designer and generate your database and classes from there.  Alternately a lot of DDD (Domain Driven Design) folks prefer to hand code their model classes. Since those classes end up being POCO and EF has full support for POCO, it works out quite well for everyone.

T4 templates

Currently the Entity Designer generates classes from models by using EntityClassGenerator which provides little to no control over the generated classes. In 4.0 you will have T4 templates which provide for greater control over the generated classes with richer functionality like adding attributes to properties or having the generated classes implement a custom interface.

T4 templates are used not only to generate classes but also to generate tables if you choose to start off by building the models first. For example, by default the mapping is table-per-type which can be overridden through T4 templates to produce a table-per-hierarchy schema. You can also add prefixes to the generated table names and so on.

Self Tracking Entities and N-Tier applications

Self tracking entities are a very cool feature of the Entity Framework that allow for N-tier applications. Currently when you have entities on which you wish to track changes, the object context does that for you. However, the object context only exists in the current tier and if you decide to pass that object around to other tiers where the context is not present, you cannot track the entities any more. For example, if you pass the Person object to the UI layer which presents it to the user for modification, those modifications are not being tracked because the object context is not available in the UI tier. N-tier support can now handle this scenario quite elegantly.

One way to do this is to generate self-tracking entities as these entities have logic to track changes within. If the UI modifies the Person object’s address, the Person object will now contain information on how it’s been modified because it is self-tracking. When you pass the Person object back to the data tier, the object can easily relay what’s changed and the context can use this to persist the object.

Of course, this means that the object is not POCO any more because it has logic to track changes within itself so there is yet another approach. You can pass the POCO object back to the Entity Framework and attach it to the context which can then do a diff on the object and the original object to determine what’s changed. This requires more processing obviously.

Proxy Tracking

Besides diff’ing the POCO entities or using self-tracking entities for change detection, another approach is provided by the Entity Framework 4.0 which is proxy tracking. It provides the benefits of POCO without the performance implications of doing a diff (to check what’s changed each time the POCO object is re-attached to the context) albiet it cannot be used in an n-tier scenario.

If using proxies to track changes, EF sets up proxy classes to communicate with your POCO classes. These proxy classes inherit from the POCO classes and any property getter/setter calls are still made to the POCO class but through the proxy class. This allows the Entity Framework to stay in sync with entities at all times. You do however need to be a bit careful if you are going to serialize these objects because the resulting output could be a proxy object (though easily remedied by calling something like EFPocoAdapter’s ProxiesToPoco() equivalent in EF 4.0).

Conclusion

This is going to be a big release for EF to bring it in line with the other major ORMs out there. The MVC community right now favors Linq to SQL but hopefully we’ll see more ASP.NET MVC + Entity Framework projects coming out.

1. Properties allow access modifiers on getters and setters

   public string Name { get; private set; }

   which makes the property read only publicly but read/write privately.

2. Properties can be virtual. Fields cannot be overwritten.
3. Properties can be declared in an interface. Fields cannot.
4. Properties provide fine-grained control such as data validation or simultaneously logging to a file.
5. .NET (by design) uses properties for data-binding making them necessary.
6. Changing a field to a property (or vice-versa) breaks binary compatibility, thus it’s a breaking change (requires recompilation). It may be desirable to use properties for future extensibility instead of thinking YAGNI.
7. Moving from fields to properties breaks source compatibility. For example, if you are using reflection on any of the fields which have since been changed to properties, then the code for reflection will need to be updated (PropertyInfo vs FieldInfo). Similarly, serialization will also break.
8. Since properties can contain some additional logic, they can throw arbitrary exceptions. Fields cannot.
9. Break points can be set on properties making them easier to debug. They can’t be set on fields.
10. Fields can be used for ref or out arguments but properties can’t.
11. Properties can do lazy loading

    public string Name {
        get {
            if (this._name = null) LoadName();
            return this._name;
        }
    }
    

These reasons should be good enough to persuade you to use properties instead of fields.

I came across the JSMin task for NAnt which was pretty good but not as good as YUI Compressor. I came across a .NET C# port of the JAVA based YUI Compressor and also found a custom task for it but no DLL. I threw the two together with the latest .NET port of YUI Compressor and put it up on my projects for download under NAnt task for YUI Compressor.