DEMIDS is an early paper on how to detect misuse of a database (warning: PDF loads slowly). As an overview, the paper describes a system where misuse is ‘detected’ by the use of a distance function. It attributes a set of tables or database functions as the normal domain of a user, and everything that the user accesses outside of that specified domain has some distance factor associated with it. Tables in other schema’s are viewed as being a certain distance outside of that domain, and tables in different database further still. The further away a resource is, the more likely there is misuse. It is a basic assumption that the users are sufficiently privileged to perform the access. And it is inherent with the methodology described that the system is closely coupled to the database itself, and it performs the work of detection locally.
While we have seen many papers on Intrusion Detection and Prevention for the general case, this was one of the first papers that I had seen written specifically for database misuse detection. I mean two things by this, insiders vs outsiders, and using database internals as opposed to external information. And as such, almost every patent application in the area of heuristic or dynamic database monitoring has to account for this work. Database misuse detection through Heuristics is a somewhat tough problem to crack. It’s not that we do not have the technology and the ability to detect the problems, we do.
By looking at data and meta-data, examining objects and distance, by looking at users and time, by looking at history and present activity, by looking at location and function, and any combination thereof, we can actually do a really good job at detection. The problems are two-fold, in that the way we use databases has actually changed considerably over the last 5 years, and that every company uses databases in slightly different ways. That means that both user activity changes and the business rules change. It means that deployment of a good heuristic system is more expensive as the threat model is more complex, but also that a behavior based algorithm that can adapt to environmental changes require less tweaking and have a longer shelf life than. Technically speaking, DEMIDS is a behavior based detection algorithm.
I have probably spent more time studying this paper than its author spent writing it at this point. I have spent months with various patent attorney’s attempting to explain the distance function and how this differs from other patents, applications and prior art. And months more educating patent examiners what this really means and how it differs from other claims. That is because they read this paper and they think they know what it is doing and what is being described. Pre-conceived notions are a powerful thing. And my protestations to the contrary are met with skepticism. Then I force them to work through the equations … a mathematical proof if you will … and they get random garbage rather than the numbers they were expecting. When I explain again how the distance calculation actually works and produce an expected numbers, they can barely believe it. Believe it. What is going on here is subtle and clever, albeit not particularly useful in today’s world. Still, anyone out there who is considering database misuse detection algorithms, this paper needs to be in your repertoire. Any patent work you do in this area, the examiners will send this work back to you as prior at and ask you to explain how you are novel.
I bring this up to illustrate the change in database usage has evolved considerably since 2000, as has our mindset on database security. It is a testament to how far we have come in this industry as a whole; how databases are used, the volume of information moving through them, the number of users and roles, and how we distribute and share data. When this paper was written the concept was well ahead of its time, but still has been eclipsed because of the vast amount of research and development into this field of expertise. Typical uses of the database, like anonymous connection pools or mirroring are not appropriately accounted for. Still, it is really worth a read to understand some of the early approaches to the problem of detecting authorized (re: insider) database misuse if this is a subject that interests you.
Also using this work in my dissertation proposal, have you found anything from the authors in terms of success/failure metrics of DEMIDS?
Thanks!
Rita.
Posted by: rita | November 26, 2010 at 08:39 AM