Best practice indicates that if you have a considerable amount of content within SharePoint then you should be looking to split that into multiple site collections. I’m not going into why that’s important in this post, but it does raise a question around one of the most powerful features that was introduced within MOSS 2007 – aggregation.
[Remember SharePoint Portal Server 2003 when we didn’t have the ability to easily aggregate data from throughout the site! Now most of the solutions I work on have some form of aggregation.]
The downside to multiple site collections is that out-of-the-box there is no easy way to use the fantastic Content Query Web Part to pull data from those multiple collections. Here are a few methods that I’ve seen which I think are good solutions:
1) Search! The easiest and probably most reliable way is to use the inbuilt search functionality which already easily copes with multiple site collections to index your data. Using the out-of-the-box search web parts configured to a fixed query (e.g. return all documents of content type x) plus a little XSL to style the results and you have a pretty good imitation of aggregation. I say imitation because obviously this is dependent upon the frequency of your search crawl and therefore not real-time aggregation. A workable solution for most scenarios however.
2) I’ve just seen (the reason for this post) that Andrew Connell and Tod Baginski have produced the ‘Content Monster Web Part‘, which can aggregate data from multiple site collections and even from external content sources. A quick look at this indicates that it could be very useful indeed. They kindly make their code available so that you can see just how they do it.
3) A 3rd party product by the Lightning Tools team. These guys, who have some great BDC offerings, also have the ‘Lightning Conductor Web Part‘ which allows for it to be configured very similarly to the CQWP but with the added bonus of choosing site collections to aggregate from. Definitely worth a look.
Search is my preferred option if the solution doesn’t require real-time aggregation or complex filtering (though this could be done with search too in most cases).