CBC News In Depth: Technology: Data mining - Fighting crime with databases: What's the connection between a lower crime rate, the price of shampoo at your local big box retailer, the stock market and next quarter profits?

The answer, surprisingly, is data mining, combing through statistics and details to establish a relationship between the numbers and other information.

Data mining isn't new, but a convergence of faster and cheaper hardware, robust and flexible software and deflating storage costs are making it possible to not only crunch more numbers at a time but to get the answers almost instantaneously.

The upshot is, managers can make mid-course adjustments in their strategies to either limit losses or maximize profits. Until recently, only global scale enterprises like Wal-Mart could afford the technology — methodically tracking which items on which shelves sold at which stores and at what time, leveraging the data to ensure product was constantly replenished at the right rate for local sales and thus controlling costs on everything from shampoo to baby formula.

What's good for Wal-Mart is good for the nation and that business intelligence strategy is now turning up all over the place.

"The issue for a lot of companies is how to get more value from the data they have," notes Anne Milley, director of analytics for SAS, a major player in supplying the systems behind the technology. "The answers are all in the numbers, and in doing so the ripples are spreading out to crime detection, anti-terrorism, air quality monitoring and surveillance."

From mapping the human genome, to bringing drugs to market faster, to cutting costs in servicing ATMs, to better predicting hurricanes, the need to capture more data and compare it to a wider range of variables is growing exponentially, says Milley. And public policy is seeing some benefits.

Policing is a prime example. Earlier this year a bedazzled Rodney Munroe, chief of the Richmond, Va. police service, accepted the 2007 Business Intelligence Award for Excellence from Gartner, a leading information technology research and advisory company, at a black tie Chicago event.

The award usually goes to innovative business applications, but judges were impressed with how Munroe turned analytics into a formidable crime-fighting tool.

(Read Original Article - Via CBC News In Depth .)