Understanding WebSnap


Last update: 2001.11.06.

This document provides an in-depth review of WebSnap, as implemented in Borland Delphi 6, Enterprise Edition.

This is currently work-in-progress.

What is WebSnap?

WebSnap is a set of technologies you can use to create web applications. It is a superset of the InternetExpress components, which are still available in the Delphi component palette.

WebSnap provides a host of new features, including the use of server-side scripting languages, better debugging support, and enabling a modular approach to developing applications.

Why should I use WebSnap?

For starters, WebSnap allows you to use many units to hold your components. This allows many developers to work on the same project, and to better organize its content. For large websites, this is essential.

WebSnap also makes it very easy to change the application after it has been deployed. While a bit of clever hacking could accomplish a similar effect using InternetExpress, WebSnap makes it a snap. Sorry, this is the last time (I hope) I'll be making this pun.

WebSnap includes a number of performance enhancements, such as better cache management and dynamic registration of page and data modules.

WebSnap allows you to use a scripting language on the server, and to expose your own components to this script. By doing this, you can easily customize your look-and-feel without rebuilding the application, after you have deployed your application.

Although this is not specific to WebSnap, there is a new page debugger for Delphi 6. It basically replaces your web server, and logs requests and responses, traces performance, and allows you to break into your code and debug it as usual.

Why should I not use WebSnap?

WebSnap is not supported in versions prior to Delphi 6. It is also much more complex than InternetExpress, and bugs seem to crop up fairly frequently.

Thankfully, a host of bugs were addressed by a patch issued by Borland.

How does the debugger work?

When you create a new WebSnap project, you have to decide what kind of project you want, based on the kind of server you will run it. One of the new options (Apache is the other one) will register your project as a COM component - an external, executable component, to be exact.

The debugger works by reading in requests, and then calling your COM component. Because support for debugging COM executable components is already very well supported (just run the application, and you are set to go), this enables developers to use all the features they are familiar with, rather than resorting to tricks such as logging activity to files or sending messages to the interactive user.

Support for the COM Web Application can be found in the ComApp unit. The request handler (more on this later) is found in the ComHTTP unit.

To launch the debugger, select Tools | Web App Debugger from the Delphi menu.

How are requests processed?

All WebSnap applications share the same request processing architecture.

First, a request is received by the application. The way this is done depends on the kind of web server the application was meant to run on. CGI web applications have a certain way, Internet Information Server web applications receive information in a different way, and so on.

The request is encapsulated in a TWebRequest object. TWebRequest is an abstract class, declared in the HTTPApp unit, and allows web applications to access information about the HTTP request without being dependant on the web server. This is what allows you to build your application for the Web App Debugger, and then change only the project unit to rebuild everything for, say, Internet Information Server.

This request will be given to a TWebRequestHandler object. This object will activate the web context, and look in all registered modules for an object that supports the IWebAppServices interface, and then ask this object to actually handle the request.

Let's stop here for a second. First, what do I mean by web context? WebContext is a global-like function in the WebCntxt unit. What do I mean by global-like function? This is a function that is meant to be used as a variable. Typically, invoking this function will create an object on demand, access a thread variable, or request the real value from somewhere else. This pattern is used throught WebSnap, so it's important to introduce this now. Anyway, WebContext allows any code to access the current request and response objects; this simplifies handling enormously, as there is no need to keep passing the objects around.

What is IWebAppServices? This is an abstraction of services an application should be able to provide. Using interfaces like this is another common pattern found in WebSnap. By using interfaces, any object, no matter where it is in the class hierarchy, can support this services. The interface is defined in HTTPApp.

One last word of advice: do not rely on WebContext being assigned, and do not rely on the Request and Response properties being assigned. During design-time, for example, you may find that they have not been initialized - and you do not want to have your code be unusable at design-time just because. Checking for not nil before using the objects will make your objects more robust.

What happens when a request gets to the application services?

This depends entirely on the implementation. You could write your own component implementing this interface, and then you would be able to customize your response from the very beginning. This is what you gain as a tradeoff for WebSnap's complexity: a lot of flexibility.

If you are interested in using the standard implementation, then you will want to use the TWebAppComponents component, which is usually created automatically along with your home page by the wizard. This component is actually a published version of TCustomWebAppComponents , which in turn is descended from TMultiModuleWebAppServices . This last class is the one that implements IWebAppServices , along with other interfaces we will examine later on. You can find the components in the WebDisp unit.

If you are really curious, like me, you will probably want to take a peek at the source code. Here, we will find another very common pattern, for example in the InitContext method in TMultiModuleWebAppServices . All this method does is call the virtual ImplInitContext . This method, because it is virtual, can be overriden by any descendant. This is a pattern you will find very frequently in stock interface implementations.

Anyway, what happens to the request here? First, the BeforeDispatch event is invoked. If you send a response in this event, no further processing takes place. Otherwise, the adapter dispatcher, action dispatcher, and page dispatcher are given a chance to handle the request (in that order). If after all of this, the response has still not been sent, the AfterDispatch event is invoked.

And where do these other dispatchers come from? They are all set from the component's properties. The Adapter dispatcher will typically dispatch actions based on hidden fields and actions (more on these later). The IWebDispatchActions interface is not usually used; it is similar to the InternetExpress dispatcher, which matched a part of the URL and used the HTTP method to select an action to execute or component to get content from. Finally, the page dispatcher is used to select a specific page; this is specially useful for web applications with multiple modules.

Each of these dispatchers have their own way of dealing with requests. We will now study them in some more detail.

How does the adapter dispatcher handle requests?

The adapter dispatcher is typically an instance of the TAdapterDispatcher class, implemented in the WebDisp unit. Again, using a typical pattern, the class itself is nothing but a published version of TCustomAdapterDispatcher . This class handles requests by retrieving "dispatch parameters" from the request.

What are these "dispatch parameters"? They are parameters encoded with special, hard-coded names in the web request. They are accessed through the TAdapterDispatchParams class (or TAdapterDispatchParams2 if you have patched Delphi). The hard-coded parameter names can be found in the const section at the end of the interface section of the WebDisp unit.

In addition to these parameters, other parameters may be extracted. If you wish to have a parameter extracted by the adapter dispatcher, you can use the RegisterAdapterRequestIdentifier routine from the WebDisp unit.

If among the parameters found, a request handler is specified, it should implement the IAdapterRequestHandler. This object will have its request context set, and the TCustomAdapterDispatcher BeforeDispatch event is invoked. If the request is not handled, the IAdapterRequestHandler object will be given a chance to handle the request. Whether it handles it or not, the AfterDispatch event is invoked (note that this is not consistent with, for example, the application service's handling of post-requests).

And here there is a very good question begging to be asked. How are variable actions found? In the request, all we have is a string. This magic trick is performed by an instance of the TVariableLookup class, declared in the WebScript unit. Instances of this class can look for an object, given its identifier and a search context (nil specifies application global context). The implementation is not very difficult to understand - the identifier is parsed as an object identifier. For example, MyObject.MyProperty references MyProperty in the MyObject context. The lookup can find variables in adapters on modules, or variables "global" to the given context (which are not real Object Pascal globals).

How does the action dispatcher handle requests?

To be done.

How dows the page dispatcher handle requests?

As you might have come to expect by now, TPageDispatcher is a published version of TCustomPageDispatcher, implemented in the WebDisp unit.

This dispatcher handles requests by looking for a page, given the path in the URL. If none is specified, the default is retrieved from a property, or the first available page in the application module (the application module is the web module that provides application services).

Once the page name has been determined, the OnBeforeDispatchPage event is fired. If the request is not handled, then some checks are performed - the user must be logged in if required, and access must be granted. If all goes ok, then the web module with the page is retrieved; if it does not exist, it is created on-demand. Note that the OnAfterDispatchPage event is fired only if the page was given a chance to be generated.

Where is the web module created from? The web request handler (the very first one to get its hands on the web request, even before the application sevices, remember?) holds a list of web module factories. If you examine the generated code whenever you add a new page or application module, you will see that in the initialization section of the unit, a factory is added to the web request handler, if available. The list, an instance of TWebModuleList, holds a cache of modules and creates the on-demand. You can find the list in the WebReq unit.

How does a web module handle the request, once it receives it? It is queries for its IPageResult interface, declared in the SiteComp unit, and the implementation is invoked to handle the request. Web page modules, whether they are normal page modules or application modules, implement IPageResult (it is implemented in the TCustomWebPageModule class, in the WebModu unit). In fact, the interface is implemented by a property of the class, of type TSitePageModuleHelper. The implementation for ImplDispatchPage will raise the OnBeforeDispatchPage event, and if not handled, will try to retrieve the page template (typically from the file locator service from the application services), use the module's PageProducer if the template could not be found, and finally invoke the OnAfterDispatchPage event.

Why all these TCustomWhatever?

By using TCustomWhatever components, and having the regular component be merely published versions of these, Borland gave developers the huge advantage of being able to roll your own version of a component, while restricting the published properties and/or events. Typically, you will find that the events are invoked from virtual methods. If you wish to have a component that performs some action on a method invocation, you merely have to override the method - and then decide to fire the event or not, as you please.

Because developers get familiar with events fairly quickly, you will find that rolling your own components is not very difficult.

What can Adapters do?

Adapters are objects that implement a lot of interfaces to work correctly. The idea is that adapters allow you to access actions and fields from scripts and HTTP requests. They are very flexible, and as you might expect, quite complex.

The root of adapters is the TCustomAdapter class, which inherits from TComponent. This class implements many interfaces. Here's a list, with a brief description of what the interface is used for.
  1. IIdentifyAdapter. This empty interface does nothing. The interface definition is in the WebAdapt unit. Declaring this interface as supported, however, identifies the object as an adapter.
  2. IWebVariableName. This interface, declared in the HTTPProd unit, provides a name for the adapter to be referenced as a variable. It uses the Name property.
  3. IWebVariablesContainer. This interface, declared in the HTTPProd unit, provides access for variables within the adapter.
  4. INotifyList. This interface, declared in the SiteComp unit, provides methods to add and remove objects to be notified of changes in the adapter.
  5. IGetAdapterErrors. This interface, declared in the SiteComp unit, enabled the adapter to provide a list of errors.
  6. IGetAdapterErrorsList. This interface, declared in the SiteComp unit, is pretty much the same thing as the IGetAdapterErrors interface, but it returns an interface rather than an object.
  7. INotifyWebActivate. This interface, declared in the WebComp unit, allows the adapter to receive notifications about the activation and deactivation of the request processing (remember how the request handler setup the web context on activation and tore it down for deactivation?)
  8. IAdapterEditor. This interface, declared in the WebAdapt unit, allows the adapter to choose whether certain actions or fields can be added to it.
  9. IGetScriptObject. This interface, declared in the SiteComp unit, allows the adapter to provide an IDispatch interface for the adapter. IDispatch is an interface declared as part of the COM Automation specification. TCustomAdapter creates a TAdapterWrapper, declared in the AutoAdapt unit, to implement this.
  10. IGetAdapterFields. This interface, declared in the SiteComp unit, allows the adapter to provide an object with information about its fields. The internal object used is an instance of TAdapterFields .
  11. IGetAdapterActions. Same thing as IGetAdapterFields , but it works with actions, and the object used is an instance of TAdapterActions.
  12. IIteratorSupport. This interface, declared in the SiteComp unit, allows the adapter to provide some sort of iteration. Specifically, TCustomAdapter tries to implement iteration through the OnIterateRecords event.
  13. IClearAdapterValues. This interface, declared in the WebAdapt unit, allows the adapter to clear its values. TCustomAdapter does this by setting the EchoActionFieldValue property of every visible fields to False. This has the effect of requesting that fields take their value from wherever it is that they usually take them (for example, a database), rather than echoing the values given by the request object (which is done, for example, when an error occurs and the user would want to modify his erroneous data).
  14. IEchoAdapterFieldValues. This interface, declared in the WebAdapt unit, allows users of the adapter to set the  EchoActionFieldValue of all fields at once.
  15. IAdapterAccess. This interface, declared in the SiteComp unit, allows access to the adapter to be checked. TCustomAdapter will allow the EndUser property of the WebContext to decide, if the property on the component is not empty and the WebContext has been initalized (see what I advised a bit earlier? - good developers check for variables being assigned before using them).
  16. IGetAdapterHiddenFields. This interface, declared in the SiteComp unit, allows access to the adapter hidden fields.
  17. ICreateActionRequestContext. This interface, declared in the WebAdapt unit, is used by clients to allow the adapter to prepare to execute an action. Actions use this interface to allow the adapter to prepare. MORE INFORMATION IS NEEDED ON THIS.
  18. IWebDataFields. This interface, declared in the SiteComp unit, allows the adapter to provide access to data fields.
  19. IWebActionsList. This interface, declared in the SiteComp unit, allows the adapter to provide access to a list of actions.
  20. IAdapterNotifyAdapterChange. This interface, declared in the WebAdapt unit, allows clients to notify the adapter that notifications should be sent about changes. TCustomAdapter will notify all the registered objects added to its notification list through INotifyList, of the change to itself.
  21. IIteratorIndex. This interface, declared in the SiteComp unit, allows the adapter to advise clients on whether it is being iterated and what is the current index interator.
Whew... that's quite a list. Adapters can do at least all the things supported by their interfaces. Descendants may do other things, too.

Basically, adapters are objects that can be used from server-side script, or from within your Object Pascal code, and that may provide fields with data, actions to execute code, and the capability to iterate over sets of data (typically used for database records).

How do adapters work?

Adapter fields work by through their GetValue method, part of the IWebVariableName interface. The standard TAdapterField component works by firing the GetValue event (unless the adapter is echoing back the user's values). A number of other events are also exposed to manage the field label and other attributes.

An interesting point is that adapter work when invoked through the server-side scripting engine. The component that handles the interface to the scripting engine is, for fields, TAdapterFieldWrapper (found in the AutoAdapt unit), implementing the IAdapterFieldWrapper interface, declared in the WebScript_TLB unit. Note that WebScript.tlb is one of the files that you must register on the server when you deploy a WebSnap application.

Note: If you take a peek at the WebScript_TLB unit, you will notice that it declared a bunch of wrapper interfaces for standard objects. This is because the server-side scripting engine uses Automation to reference objects. However, Automation objects need to implement certain interfaces (see IDispatch if you are morbidly curious), which are normally delegated to helper objects that read the information off a type library. However, because there is a need for more dynamic things than just CoClasses and static interfaces, the WebAuto and AutoAdapt units work to provide further services. SiteComp also pitches in, declaring the automation-happy TScriptObject and TScriptComponent classes. If you are looking for more information on this, take a look at the How does scripting work? section.

When you are using an object of type TAdapterPageProducer instead of writing your own script directly to access your fields, it is pretty much the same thing. Remember that the web elements in the object will generate script at runtime, after inspecting the object (using traditional Object Pascal), and only when the page content is evaluted by the scripting engine will the value be retrieved.

Anyway, so far, we only got half of the story told. How do adapters handle actions?

Adapter actions are usually invoked by the adapter dispatcher. The ones developers add to TAdapter components are of type TCustomAdapterAction , implemented in the WebAdapt unit, and simply forwards all behaviour to the defined events, and adds support for setting the action name through the IWebSetActionName interface.

The direct ancestor, TBaseAdapterAction, is much more interesting. This component also defines a host of interfaces, which provide it with a very rich functionality. The following are the interfaces introduced and implemented in TBaseAdapterAction.
How is the request executed? In the HandleRequest method (invoked through the IAdapterRequestHandler interface), the action will check permissions on it, and then invoke DoBeforeExecuteActionRequest (which by default calls the parent Adatper's OnBeforeExecuteAction and if that doesn't handle it then the OnBeforeExecute event), ImplExecuteActionRequest and DoAfterExecuteActionRequest (which calls both its own after event and its parent's, in this order). Note that DoAfterExecuteActionRequest will be executed even if an exception is raised in ImplExecuteActionRequest. DoBeforeExecuteActionRequest can halt processing by setting the Handled property on the IActionRequest interface.

After the request has had a go through DoBeforeExecuteActionRequest , ImplExecuteActionRequest and DoAfterExecuteActionRequest , something similar happens to retrieve the action response if the IActionResponse interface is not marked as handled. The DoBeforeGetExecuteActionResponse , ImplGetExecuteActionResponse and DoAfterGetExecuteActionResponse virtual methods are invoked, in that order. DoBeforeGetExecuteActionResponse will also try to fire events, ImplGetExecuteActionResponse will set the ExecuteStatus property and try to respond with its own component page by default (that is, if the RespondWith on the reponse is undefined).

Note that the ExecuteActionRequest family of methods works with the Handled property on the IActionRequest interface, and the GetExecuteActionResponse works with Handled property of the IActionResponse interface.

For a sample of how all of these can be used, take a look at how dataset adapters work.

This does not mean, however, that you cannot (invoke 'em from code? how? )

For more information on what is available through adapters, take a look at the WebSnap server-side scripting reference, available through the Delphi help system. In particular, look at the Object types topic and subtopics, as they provide information on how to access the objects you create yourself.

How do dataset adapters work?
The component you will surely use is TDataSetAdapter, declard in the DBAdapt unit. This component is a custom adapter, designed to adapt a regular TDataSet to server-side scripting and the WebSnap framework.



How does scripting work?

Scripting is enabled by the TBasePageProducer class in the HTTPProd unit. In the ContentFromStream method, which is used to retrieve the page contents, an abstract scripting class is invoked to process the content stream.

The abstract script procedure in HTTPProd, but the COM-based implementation for Windows is in fact implemented in the WebScript unit, by the TScriptProducer class. This class also implements two interfaces, declared in HTTPProd: IScriptProducer and IScriptContext. The first inherits from the second, and is used to retrieve error information and manipulate the parsing.

TScriptProducer will create an Active Script class, and use the interfaces declared in the AscrLib unit to request that the content is parsed and executed (as a side effect of being evaluated). In fact, the expression is not evaluated directly, but rather by a helper TScriptSite object (or a class compatible with TScriptSite ). This last object, will establish itself as the 'site' of the scripting execution, effectively providing context for the parsing. This is done by using the IActiveScript interface to add named items, such as Response or Request. For example, during testing, I found the following 12 named items being used: Response, Producer, HTML_, ApplicationAdapter, MyAdapter (this is a test adapter running on the active page), Request, Page, Modules, Pages, Application, EndUser and Session.

When the scripting engine finds a reference to one of these named objects, it will call out to it through Automation interfaces to retrieve values or execute actions. But how does the script site object know what variables to add and what objects should handle calls through them?

The script site is created by a TScriptObjectFactory, implemented in the WebScript unit. This object will add, just after created the object, all the global references: Response, Producer, etc. Then the objects found in the web context, as retrieved through the IWebVariablesContainer interface, will be added. These variables are typically the adapters found in the currently executing web module. In fact, the objects are not added directly - an Automation-aware adapter wrapper is created for the object, and it is this wrapper that gets added to the script site, and, eventually, to the IActiveScript implementation.

Yada - yada
To be yadded.