• Application problems are not necessarily easier solved by logs from a software layer that sits at the lower end end of the architecture pyramid (the database being the foundation of the application and all that). The database generally does not provide all the context needed to pinpoint what is really happening. You need to know from what webpage code was invoked and what did the user submit in his form, and which user was this and what did that user do before, etc.

    Even with proper logging and this rich context available, a person reading the error must interpret it and understand the processes that the application uses. Something technical might go wrong (a deadlock or log space is full), but without understanding the processes of the application it still would be mostly useless.

    Logging should be done and handled by the entire tier of the software pyramid to give as much context to an error as possible. This is why there most certainly has to be something for it build into the top-most application layer. The consumers of this information are people that KNOW the application well, not just any server administrator. Those have to use their own technical application unspecific logs to work independently. Even better they should have monitoring software that provide alarms long before most technical issues become apparent to applications.

    In practice we have expose errors in two different ways.

    1. We let an application log to a application specific folder that stores an index and a human readable error page (html) that exposes as much relevant information as possible. Including CGI, form and session variables as well as the executed query and its parameters. It also adds code snippets based on the stack trace thus we can instantly see what code was actually being executed. We have a error viewer that reads the index and groups similar errors and at which times these occurred. You can zoom in onto these to get to the html page of an individual error.

    2. Essentially the same as option 1, but this is server wide for application that do not have their own error handling.

    We have made preparations for gathering all of these errors into a single place, with a single more advanced viewer and an alert system. We just haven't found the time to finish that and use the current logs and viewers to analyze problems as we become aware of them.