We have earlier discussed about Web Deployment and Web Packaging quite a bit, today I wanted to dive into web.config transformation. If you would like to check out the other topics please read through the earlier blog posts below:

• Web Deployment with VS 2010 and IIS
• Web Packaging: Creating a Web Package using VS 2010

• Web Packaging: Creating web packages using MSBuild

• How does Web Deployment with VS 10 & MSDeploy Work?

• Installing Web Packages using Command Line

Usually web applications go through a chain of server deployments before being finally being deployed to production environment. Some of these environments can be Developer box (Debug), QA Server, Staging/Pre-Production, Production (Release). While transitioning between these environments various settings of the web application residing in web.config file change, some of these settings can be items like application settings, connection strings, debug flags, web services end points etc.

VS10’s new web.config transformation model allows you to modify your web.config file in an automated fashion during deployment of your applications to various server environments. To help command line based deployments, Web.Config transformation is implemented as an MSBuild task behind the scene hence you can simply call it even outside of deployment realm.

I will try to go through below steps to explain web.config transformation in detail

1. Creating a “Staging” Configuration on your developer box

2. Adding a “Staging” Web.Config Transform file to your project

3. Writing simple transforms to change developer box connection string settings into “Staging” environment settings

4. Generating a new transformed web.config file for “Staging” environment from command line

5. Generating a new transformed web.config file for “Staging” environment from VS UI

6. Understanding various available web.config Transforms and Locators

7. Using Web.config transformation toolset for config files in sub-folders within the project

Step 1: Creating a “Staging” Configuration on your developer box

Debug and Release build configurations are available by default within Visual Studio but if you would like to add more build configurations (for various server environments like “Dev”, “QA”, “Staging”, “Production” etc then you can do so by going to the Project menu Build --> Configuration Manager… Learn more about creating build configurations.

Step 2: Adding a “Staging” Web.Config Transform file to your project

One of the goals while designing web.config transformation was to make sure that the original runtime web.config file does not need to be modified to ensure that there would be no performance impacts and also to make sure that the design time syntax is not mixed with runtime syntax. To support this goal the concept of Configuration specific web.config files was introduced.

These web.config files follow a naming convention of web.configuration.config. For example the web.config files for various Visual Studio + Custom configurations will look as below:

Any new Web Application Project (WAP) created in VS10 will by default have Web.Debug.config and Web.Release.config files added to the project. If you add new configurations (e.g. “Staging”) or if you upgrade pre-VS10 projects to VS10 then you will have to issue a command to VS to generate the Configuration specific Transform files as needed.

To add configuration specific transform file (e.g. Web.Staging.Config) you can right click the original web.config file and click the context menu command “Add Config Transforms” as shown below:

On clicking the “Add Config Transform” command VS10 will detect the configurations that do not have a transform associated with them and will automatically create the missing transform files. It will not overwrite an existing transform file. If you do not want a particular configuration transform file then you can feel free to delete it off.

Note: In case of VB Web Application Projects the web.configuration.config transform files will not be visible till you enable the hidden file views as shown below:

The transform files are design time files only and will not be deployed or packaged by VS10. If you are going to xCopy deploy your web application it is advised that you should explicitly leave out these files from deployment just like you do with project (.csproj/.vbproj) or user (.user) files…

Note: These transform files should not be harmful even if deployed as runtime does not use them in any fashion and additionally ASP.NET makes sure that .config files are not browsable in any way.

Step 3: Writing simple transforms to change developer box connection string settings into “Staging” environment settings

Web.Config Transformation Engine is a simple XML Transformation Engine which takes a source file (your project’s original web.config file) and a transform file (e.g. web.staging.config) and produces an output file (web.config ready for staging environment).

The Transform file (e.g. web.staging.config ) needs to have XML Document Transform namespace registered at the root node as shown below:

<?xml version="1.0"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
</configuration>

Note: The transform web.config file needs to be a well formed XML.

Inside the XML-Document-Transform namespace two new attributes are defined. These attributes are important to understand as they drive the XML Transformation Engine.

Transform – This attribute inside the Web.Staging.config informs the Transformation engine the way to modify web.config file for specific configuration (i.e. staging). Some examples of what Transforms can do are:

• Replacing a node

• Inserting a node

• Delete a node

• Removing Attributes

• Setting Attributes

Locator – This attribute inside the web.staging.config helps the Transformation engine to exactly pin-point the web.config node that the transform from web.staging.config should be applied to. Some examples of what Locators can do are:

• Match on value of a node’s attribute

• Exact XPath of where to find a node

• A condition match to find a node

Based on the above basic understanding let us try to transform connection string from original web.config file to match Staging environment’s connection string

Let us examine the original web.config file and identify the items to replace... Let’s assume that the original Web Config file’s connection string section looks as below:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <connectionStrings>
    <add name="personalDB"
     connectionString="Server=DevBox; Database=personal; User Id=admin; password=P@ssw0rd" providerName="System.Data.SqlClient" />
    <add name="professionalDB"
     connectionString="Server=DevBox; Database=professional; User Id=admin; password=P@ssw0rd" providerName="System.Data.SqlClient" />
</connectionStrings>
....
....
</configuration>

NOTE: It is not advisable to keep connection string unencrypted in the web.config file, my example is just for demonstration purposes.

Let us assume that we would like to make following changes to web.config file when moving to staging environment

• For “personalDB” we would like to change the connectionString to reflect Server=StagingBox, UserId=admin, passoword=StagingPersonalPassword”

• For “professionalDB” we would like to change the connectionString to reflect Server=StagingBox, UserId=professional, passoword=StagingProfessionalPassword”

To make the above change happen we will have to open web.Staging.Config file and write the below piece of code

<?xml version="1.0"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
     <connectionStrings>
        <add name="personalDB"
          connectionString="Server=StagingBox; Database=personal; User Id=admin; password=StagingPersonalPassword"
          providerName="System.Data.SqlClient" xdt:Transform="Replace" xdt:Locator="Match(name)" />
        <add name="professionalDB"
         connectionString="Server=StagingBox; Database=professional; User Id=professional; password=StagingProfessionalPassword"
         providerName="System.Data.SqlClient" xdt:Transform="Replace" xdt:Locator="Match(name)"/>
       </connectionStrings>
</configuration>

The above syntax in web.staging.config has Transform and Locator attributes from the xdt namespace. If we analyze the connection string node syntax we can notice that the Transform used here is “Replace” which is instructing the Transformation Engine to Replace the entire node

Further if we notice the Locator used here is “Match” which is informing Transformation engine that among all the “configuration/connectionStrings/add” nodes that are found, pick up the node whose name attribute matches with the name attribute of <add> node in web.Staging.config.

Also if you notice web.Staging.config does not contain anything else but the connectionStrings section (i.e. it does not have <system.web> and various other sections that web.config file usually has, this is because of the fact that the Transformation Engine does not require a complete web.config file in web.staging.config. It does the merging for you thus saving you duplication of all the rest of the sections in web.config file.

Simplest Approach: If you do not mind replicating the entire web.config file in web.staging.config then you can certainly do so by copying the entire web.config content into web.staging.config and change the relevant nodes inside web.staging.config. In such a situation you will just have to put xdt:Transform="Replace" attribute on the topmost node (i.e. configuration) of web.staging.config. You will not need xdt:Locator attribute at all as you are replacing your entire web.config file with web.staging.config without Matching anything.

So far we have seen one Transform (i.e. Replace) and one Locator (i.e. Match), we will see various other Transforms and Locators further in the post but first let us understand how we can produce the Transformed web.config file for the Staging environment after using original web.config and web.staging.config.

Step 4: Generating a new transformed web.config file for “Staging” environment from command line

Open Visual Studio Command prompt by going to Start --> Program Files –> Visual Studio v10.0 –> Visual Studio tools –> Visual Studio 10.0 Command Prompt

Type “MSBuild “Path to Application project file (.csproj/.vbproj) ” /t:TransformWebConfig /p:Configuration=Staging" and hit enter as shown below:

Once the transformation is successful the web.config for the “Staging” configuration will be stored under obj -->Staging folder under your project root (In solution explorer you can access this folder by first un-hiding the hidden files) :

• In the solution explorer click the button to show hidden files
• Open the Obj folder

• Navigate to your Active configuration (in our current case it is “Staging”)

• You can find the transformed web.config there

You can now verify that the new staging web.config file generated has the changed connection string section.

Step 5: Generating a new transformed web.config file for “Staging” environment from VS UI

Right Click on your project and click Package –> Create Package

The Create Package step already does web.config transformation as one of its intermediate steps before creating a package and hence you should be able to find the transformed web.config file in the same place as described in Step 4

Step 6: Understanding various available web.config Transforms and Locators

xdt:Locators

The inbuilt xdt:Locators are discussed below.

• Match - In the provided syntax sample below the Replace transform will occur only when the name Northwind matches in the list of connection strings in the source web.config.Do note that Match Locator can take multiple attributeNames as parameters e.g. Match(name, providerName) ]

<connectionStrings>
     <add name="Northwind" connectionString="connectionString goes    here" providerName="System.Data.SqlClient" xdt:Transform="Replace" xdt:Locator="Match(name)" />
</connectionStrings>

·         Condition - Condition Locator will create an XPath predicate which will be appended to current element’s XPath. The resultant XPath generated in the below example is “/configuration/connectionStrings/add[@name='Northwind or @providerName=’ System.Data.SqlClient’ ]”

This XPath is then used to search for the correct node in the source web.config file

<connectionStrings>
      <add name="Northwind" connectionString="connectionString goes here" providerName="System.Data.SqlClient" xdt:Transform="Replace" xdt:Locator="Condition(@name=’Northwind or @providerName=’System.Data.SqlClient’)" />
</connectionStrings>

·         XPath- This Locator will support complicated XPath expressions to identify the source web.config nodes. In the syntax example we can see that the XPath provided will allow user to replace system.web section no matter where it is located inside the web.config (i.e. all the system.web sections under any location tag will be removed.)

<location path="c:\MySite\Admin" >
    <system.web xdt:Transform="RemoveAll" xdt:Locator="XPath(//system.web)">
    ...
    </system.web>
</location>

xdt:Transform

• Replace - Completely replaces the first matching element along with all of its children from the destination web.config (e.g. staging environment’s web.config file). Do note that transforms do not modify your source web.config file.
  <assemblies xdt:Transform="Replace">
      <add assembly="System.Core, Version=3.5.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089" />
  </assemblies>

·         Remove - Removes the first matching element along with all of its children
<assemblies xdt:Transform="Remove"></assemblies>

·         RemoveAll - Removes all the matching elements from the destination’s web.config (e.g. staging environment’s web.config file).

<connectionStrings>
    <add xdt:Transform="RemoveAll"/>
</connectionStrings>

·         Insert - Inserts the element defined in web.staging.config at the bottom of the list of all the siblings in the destination web.config (e.g. staging environment’s web.config file).

<authorization>
     <deny users="*" xdt:Transform="Insert"/>
</authorization>

·         SetAttributes - Takes the value of the specified attributes from the web.staging.config and sets the attributes of the matching element in the destination web.config. This Transform takes a comma separated list of attributes which need to be set. If no attributes are given to SetAttributes transform then it assumes that you would like to Set all the attributes present on the corresponding node in web.staging.config
<compilation batch="false"xdt:Transform="SetAttributes(batch)">

   …

   </compilation>

·         RemoveAttributes - Removes the specified attributes from the destination web.config (i.e. staging environment’s web.config file). The syntax example shows how multiple attributes can be removed.

<compilation xdt:Transform="RemoveAttributes(debug,batch)">
</compilation>

• InsertAfter (XPath) - Inserts the element defined in the web.staging.config exactly after the element defined by the specified XPath passed to “InsertAfter()” transform. In the syntax example the element <deny users="Vishal" />will be exactly inserted after the element <allow roles="Admins" /> in the destinationXML.

<authorization>
     <deny users="Vishal" xdt:Transform="InsertAfter(/configuration/system.web/authorization/allow[@roles='Admins'])” />

</authorization>

• InsertBefore (XPath) - Inserts the element defined in the web.staging.config exactly before the element defined by the specified XPath passed to “InsertBefore()” transform. In the syntax example the element <allow roles="Admins" />will be exactly inserted before the element <deny users="*" />in the destinationXML.

<authorization>
      <allow roles=" Admins" xdt:Transform="InsertBefore(/configuration/system.web/authorization/ deny[@users='*'])" />
</authorization>

Some advanced points to note:

• If the Transformation Engine does not find a xdt:Transform attribute specified on a node in web.staging.config file then that node is ignored for Transformation and the Tranformation engine moves ahead traversing the rest of the web.staging.config.

• A xdt:Transform attribute on a parent can very easily impact child elements even if there is no Transform specified for child e.g. If xdt:Transform=”Replace” is put on <system.web> then everything underneath <system.web> node will be replaced with the content from web.staging.config

• It is completely valid to place xdt:Locators attributes on arbitrary nodes inside web.staging.config just for filtering purposes. xdt:Locator does not need to be accompanied with xdt:Transform attribute. (great example here is <location> tag which might just be used for filtering… The example code here would be:

<location path="c:\MySite\Admin" xdt:Locator="Match(path)">>
        <system.web>
          ... Bunch of transforms written under here will
          .... only apply if location path = C:\MySite\Admin
        </system.web>
</location>

Step 7: Using Web.config transformation toolset for config files in sub-folders within the project

All of the above discussion directly applies to any web.config file present in sub folders of your project (e.g. if you have a separate web.config file for say “Admin” folder then VS 10 will support transforms for them too). You can add transform files within sub-folders and use the same packaging functionality mentioned in all of the above steps to create transformed web.config files for web.config files specific to the sub folders within your project.

I think this has become a rather long post; but I hope it helps!!

Vishal R. Joshi | Program Manager | Visual Studio Web Developer

Posted by: Sue Loh

I’d like to explain a little more about memory management in Windows CE.  I already explained a bit about paging in Windows CE when I discussed virtual memory.  In short, the OS will delay committing memory as long as possible by only allocating pages on first access (known as demand paging).  And when memory is running low, the OS will page data out of memory if it corresponds to a file – a DLL or EXE, or a file-backed memory-mapped file – because the OS can always page the data from the file back into memory later.  (Win32 allows you to create “memory-mapped files” which do or do not correspond to files on disk – I call these file-backed and RAM-backed memory-mapped files, respectively.)  Windows CE does not use a page file, which means that non file-backed data such as heap and thread stacks is never paged out to disk.  So for the discussion of paging in this blog post I’m really talking only about memory that is used by executables and by file-backed memory-mapped files.

It’s relatively easy to guess how the OS decides when to page data in to memory – it doesn’t page it in until it absolutely has to, when you actually access it.  But how does the OS decide when to remove pageable data from memory?  Ahh, that’s the question!

The Paging Pool and How It Works

Back in the old days of CE 3.0 or so (I’m not sure) – Windows CE did not have a paging pool.  What that means is that the OS had no limit on the number of pages it could use for holding executables and memory-mapped files.  If you ran a lot of programs or accessed large memory-mapped files, you’d see memory usage climb correspondingly.  Usage would continue to go up until the system ran out of memory.  Other allocations could fail; memory would appear to be nearly gone when really there was actually a lot of potential to free up space by paging data out again.  Until finally when the system hit a low memory limit, the kernel would walk through all of the pageable data, paging everything (yes, everything) out again.  Then suddenly there would be a lot of free memory, and you’d take page faults to page in any data you’re still actually using.

The algorithm is simple, but it has a few bad effects.  First, a bad effect of the simple paging algorithm was, obviously, that the system could encounter preventable RAM shortages.  Also, it was really tough for applications or tools to measure free memory – where “free” includes currently-unused pages plus “temporary” pages that could be decommitted when necessary.  Conversely, it was difficult for users to determine how much of an application’s memory usage is fixed in RAM vs. “temporary” pageable pages.   Even today it is tough to answer the question “how much memory is my process using?” in simple terms without diving into explanations of paging, cross-process shared memory, etc.  Another possible problem you can encounter when there’s no paging pool is that the rest of the system can take up all of the free memory, and leave you thrashing over just a few pages.

So we introduced the paging pool.  The purpose of the paging pool is to serve as a limit on the amount of memory that could be consumed by pageable data.  It also includes the algorithm for choosing the order in which to remove pageable data from memory.  Pool behavior is under the OEM’s control – Microsoft sets default parameters for the paging pool, but OEMs have the ability to change those settings.  Applications do not have the ability to set the behavior for their own executables or memory-mapped files.

Up to and including CE 5.x, the paging pool behavior was fairly simple.

·         The pool only managed read-only pageable data.  Executable code is read-only so it used the pool, and so did read-only file-backed memory-mapped files.  Read-write memory-mapped files did not use the pool, however.  The reason is that paging out read-write data can involve writing back to a file.  This is more complicated to implement and requires more care to avoid file system deadlocks and other undesirable situations.  So read-write memory-mapped files had no memory usage limitations and could still consume all of the available system RAM.

·         The pool had one parameter, the size.  OEMs could turn the pool off by setting the size to 0.  Turning off the paging pool meant that the OS did not limit pageable data – behavior would follow the pattern described above from before we had a paging pool.  Turning on the pool meant that the OS would reserve a fixed amount of RAM for paging.  Setting the pool size too low meant that pages could be paged out too early, while they’re still in use.  Setting the pool size too high meant that the OS would reserve too much RAM for paging.  Pool memory would NOT be available for applications to use if the pool was underutilized.  A 4MB pool took 4MB of physical RAM, no matter whether there was only 2MB of pageable data in use or 100MB.  Setting the size of the pool was a tricky job, because you had to decide whether to optimize a typical steady-state situation with several applications running (and judge how much pool those applications would need), or optimize “spike” situations such as system boot where many more pages were needed for a short period of time.

·         The kernel kept a round-robin FIFO ring of pool pages: the oldest page in memory – the earliest one to be paged in – was the first one paged out when something else needed to be paged in, regardless of whether the oldest page was still in use or not.

So the short roll-up of how the paging pool worked up through CE 5.x is that the paging pool allowed OEMs to set aside a fixed amount of memory to hold read-only pageable data, and it was freed in simple round-robin fashion.

In CE 6.0, the virtual memory architecture changes involved major rewriting of the Windows CE memory system, including the paging pool.  The CE 6.0 paging pool behavior is still fairly simplistic, but is a little bit more flexible.

·         CE 6.0 has two paging pools – the “loader” pool for executable code, and the “file” pool which is used by all file-backed memory-mapped files as well as the new CE 6.0 file cache filter, or “cache manager.”  This way, OEMs can put limitations on memory usage for read-write data in addition to read-only data.  And they can set separate limitations for memory usage by code vs. data.

·         The two pools have several parameters.  Primary of these are target and maximum sizes.  The idea is that the OS always guarantees the pool will have at least its target amount of memory to use.  If memory is available, the kernel allows the pool to consume memory above its target.  But when that happens, it also wakes up a low-priority thread which starts paging data out again, back down to slightly below the target.  That way, during busy “spikes” of memory usage, such as during system boot, the system can consume more memory for pageable data.  But in the steady-state, the system will hover near its target pool memory usage.  The maximum size puts a hard limit on the memory consumption – or OEMs could set the maximum to be very large to avoid placing a limit on the pool.  OEMs can also get the old pre-CE6 behavior by setting the pool target and maximum to the same size.

·         Due to the details of the new CE6 memory implementation, the FIFO ring of pages by age was not possible.  The CE6 kernel pages out memory by walking the lists of modules and files, paging out one module/file at a time.  This is no better than the FIFO ring, but still leaves us potential for implementing better use-based algorithms in the future.

There are some more details in our documentation under “Paging Pool” and “Paging Pool: Windows CE 5.0 vs. Windows Embedded CE 6.0.”

Overall, enabling the paging pool means that there is always some RAM reserved for code paging and we will be less likely to reach low-memory conditions.  In general it's better to turn on the paging pool because it gives you more predictable performance, rather than occasional long delays you’d hit when cleaning up memory when you run out.  But it does need to be sized based on the applications in use, which leads to my next point...

Choosing a Pool Size

In Windows CE (embedded) 5.0, the pool is turned off by default.  In Windows Mobile, the pool is turned on and set to a default size chosen by Microsoft.  I believe it varies between versions, but is somewhere in the neighborhood of 4-6 MB.  In CE6, the loader pool has a target size of 3MB and the file pool has a target size of 1MB.  Only the OEM of a device can set the pool size; applications cannot change it.

So how do you decide on the right pool size for your platform?  I’m afraid it’s still a bit of a black art.  :-(  There aren’t many tools to help.  You can turn on CeLog during boot and see how many page faults it records.  You can see the page faults in Remote Kernel Tracker, but in truth that kind of view isn’t much help here.  The best tool I know is that readlog.exe will print you a page fault report if you turn on the “verbose” and “summary” options.  If you get multiple faults on the same pages, your pool may be too small (you may also be unloading and re-loading the same module, ejecting its pages from memory, so look for module load events in the log too).  If you don’t get many repeats, your pool may be bigger than you need.  In CE6 you can use IOCTL_KLIB_GET_POOL_STATE to get additional information about how many pages are currently in your pool and how many times the kernel has had to free up pool pages to get down to the target size.  There aren’t any tools like “mi” that query the pool state, so you’ll have to call the IOCTL yourself.  On debug builds of the OS, there is also a debug zone in the kernel you can turn on to see a lot of detail about paging and when the pool trim thread is running.  But CeLog is probably a better choice to collect all of that data.

As I already mentioned, as of CE6 you can set separate “target” and “max” values for the paging pools.  I don’t really like the semantics of having a “max” – it isn’t dependent on the other usage or availability in the system.  If some application takes most of the available memory in the system, you’d want the pool to let go of more pages.  If you have a lot of free memory, and some application is reading a lot of file data, you’d want the pool to grow to use most of the available memory.  We supported the “max” as an option to limit the pool size, but I’m starting to think the best idea is to set your max to infinity, to let the pool grow up to the size of available memory.  We’ll still page out down to the target in the background.  I’d have liked to add more sophisticated settings like “leave at least X amount of free memory” but that’s quite difficult to implement.

You’ll want to examine your pool behavior during important “user scenarios” like boot or running a predefined set of applications.  If the user runs a lot of applications at once, or a really big application, or one that reads a lot of file data, they could go through pool pages pretty quickly.  There isn’t really a lot you can do about that.  We don’t even have a set of recommended scenarios for you to examine.  I wish we had more information and more tools for this, but I’ve described about all we have.

The approach I think most OEMs take is that they leave the pool at the default size until they discover a perf problem with too much paging (by profiling or otherwise observing) in a scenario that's important to users.  Then they bump it up until the problem goes away.  Not very scientific but it works, and it's not like we have any answer that's more scientific anyway.

What goes into the paging pool

This is repeating some of the information above, but in more detail – how do you know exactly what pages will use the pool and what pages won’t?  Keep in mind that paging is actually independent of the paging pool.  Paging can happen with or without the paging pool.  If you turn off the paging pool then you turn off the limit that we set on the amount of RAM that can be taken up for paging.  But pages can still be paged.  If you turn ON the paging pool then we enforce some limits, that’s all.  So this isn’t really a question of what pages can use the pool, it’s a question of what pages are “pageable.”

Executables from the FILES section of ROM will use the paging pool for their code and R/O data.  R/W data from executables can’t be paged out, so it will not be part of the pool.  Compressed executables from the MODULES section of ROM will use the pool for their code and R/O data.  If the image is running from NOR or from RAM, uncompressed executables from MODULES will run directly out of the image without using any pool memory.  Executables from MODULES in images on NAND will be paged using the pool.  (And by the way, I’m not terribly familiar with how we manage data on NAND/IMGFS so I might be missing some details here.)

Executables that would otherwise page but are marked as “non-pageable” will be paged fully into RAM as soon as they’re loaded, and not paged out again until they’re unloaded.  These pages don’t use the pool.  You can also create “partially pageable” executables by telling the linker to make individual sections of the executable non-pageable.  Generally code and data can’t be pageable if it’s part of an interrupt service routine (ISR) or if it’s called during suspend/resume or other power management, because paging could cause crashes and deadlocks.  And code/data shouldn’t be pageable if it’s accessed by an interrupt service thread (IST) because paging would negatively impact real-time performance.

Memory-mapped files which don’t have a file underneath them (a.k.a. RAM-backed mapfiles) will not use the pool.  In CE5 and earlier, R/O file-backed mapfiles will use the pool while R/W mapfiles will not.  In CE6, all file-backed memory-mapped files use the file pool.  And the new file cache filter (cache manager) essentially memory-maps all open files, so the cached file data uses the file pool.

To look at that information from the opposite angle, if you are running all executables directly out of your image – all are uncompressed in the MODULES section of ROM, and the image is executing out of NOR or RAM, then the loader paging pool is probably a waste.  You might still want to use the file pool to limit RAM use for file caching and memory-mapped files, but in that case you might want to turn off the loader pool.

Other Paging Pool Details

Someone once asked me whether the pool size affects demand paging.  It doesn’t change demand paging behavior or timing.  Demand paging is about delaying committing pages as long as possible, and it applies to pages regardless of the paging pool.  Pages can be demand paged without being part of the pool; they won’t be paged in until absolutely necessary, and then they’ll stay in RAM without being paged out.  Pool pages will be demand paged in, and may eventually be paged out again.

Another question was whether the paging pool uses up virtual address space.  Actually, no, it doesn’t.  The pool pages that are currently in use are assigned to virtual addresses that are already reserved.  For example, when you load a DLL, you reserve virtual address space for the DLL; and when you touch a page in the DLL, a physical page from the pool is assigned to the already-reserved virtual address in your DLL.  The pool pages that are NOT in use are not assigned virtual addresses.  The kernel tracks them using their physical addresses only.  The pool *does* use up physical RAM.  In CE5 it uses the whole size of the paging pool; on CE6 it consumes physical memory equal to the “target” size of the pool.  This guarantees that you have at least a minimum number of pages to page with, to avoid heavy thrashing over just a few pages when the rest of the memory in the system is taken.

Other Paging-Related Details

A related detail that occasionally confuses people is the “Paging” flag on file system drivers.  This flag doesn’t control whether the driver code itself is pageable.  Rather, it controls whether the file system allows files to be loaded into memory a page at a time or all at once.  On typical file systems like FATFS the “Paging” flag is turned on, allowing executables and memory-mapped files to be accessed a page at a time.   On other file system drivers, such as our release directory file system (RELFSD) and our network redirector, it’s turned off by default, causing executables and memory-mapped files to be read into memory all at once.  I believe the reasoning is to improve performance and minimize problems when the network connection is lost.

This flag actually derives from the original Windows CE implementation of memory-mapped files.  If the file system supported a couple of APIs, ReadFileWithSeek and WriteFileWithSeek, memory-mapped files on that file system would be pageable.  If the file system did not support those APIs, the memory-mapped files would be non-pageable, in which case they’d be read entirely into RAM at load time and never paged out until the memory-mapped file is unloaded.  The OS required pageability for special memory-mapped files like registry hives and CEDB database volumes, so file systems that did not support the required APIs could not hold these files.  (If you ask me, there is no real need to require the seek + read/write to occur in one atomic API call, so the requirement on the “WithSeek” APIs was unnecessary, but perhaps there was a good reason back in the old days.)

As I already mentioned, the new CE6 file cache also uses the paging pool.  The file cache is basically just memory-mapping files to hold the file data in RAM for a while.  The file cache is enabled by default on top of FATFS volumes.

What is web hosting? Whenever you visit a website, what you see on your web browser is essentially just a web page that is downloaded from the web server onto your web browser. In general, a web site is made up of many web pages. And a web page is basically composed of texts and graphic images. All these web pages need to be stored on the web servers so that online users can visit your website.

Therefore, if you plan to own a new website, you will need to host your website on a web server. When your website goes live on the web server, online users can then browse your website on the Internet. Company that provides the web servers to host your website is called web hosting providers.

A well-established web hosting provider sometimes hosts up to thousands of websites. For example, iPowerWeb is a popular web hosting company that hosts more than 300,000 websites. For that reason, a web hosting company need many web servers (essentially, these are computers) to ?store? the website. And all these web servers are connected to the Internet through high speed Internet connection and housed in a physical building called ?data center?. In order to guarantee all the web servers are safe, secure and fully operational all time, a data center is a physically secure 24/7 environment with fire protection, HVAC temperature control, virus detections, computer data backup, redundant power backup and complete disaster recovery capabilities.

What are the different types of web hosting? There are different kinds of web hosting companies out there with different characteristics. The main types of web hosts can be organized into the following categories:

To continue, go to A Beginners Guide to Web Hosting article.

NB: A lot of other great web hosting services articles are available at http://web-hosting.info-and-tips.com

Today I want to advance our discussions around Web Deployment in Visual Studio 10…  To catch up on the previous discussions in this series check out:

1. Web Sites Disrupted By Attack on Register.com (Washington Post) Web site host and domain name registrar Register.com has been...
2. Web Sites Disrupted By Attack on Register.com (Washington Post) Web site host and domain name registrar Register.com has been...
3. Webhosting.uk.com Announces Windows Server 2008 Web Hosting (TopHosts.com) Windows 2008 is offered for its shared and reseller Web...
4. pair Networks Celebrates 13th Year in Web Hosting (TopHosts.com) Long-standing Pittsburgh-based Web host has provided hosting for customers in...
5. pair Networks Celebrates 13th Year in Web Hosting (TopHosts.com) Long-standing Pittsburgh-based Web host has provided hosting for customers in...

1. Best Investment Web Site Trends Detailed in New Report from the Web Marketing Association (PRWeb via Yahoo! News) Updated Web Marketing Association’ Internet Standards Assessment Report includes historical...
2. Best Professional Services Web Site Trends Detailed in New Report from the Web Marketing Association (PRWeb via Yahoo! News) Updated Web Marketing Association’ Internet Standards Assessment Report Includes Historical...
3. Best B2B Website Trends Detailed in New Report from the Web Marketing Association (PRWeb via Yahoo! News) Updated Web Marketing Association’ Internet Standards Assessment Report Includes Historical...
4. Best B2B Website Trends Detailed in New Report from the Web Marketing Association (PRWeb via Yahoo! News) Updated Web Marketing Association’ Internet Standards Assessment Report Includes Historical...
5. Best B2B Website Trends Detailed in New Report from the Web Marketing Association (PRWeb via Yahoo! News) Updated Web Marketing Association’ Internet Standards Assessment Report Includes Historical...

1. UK2 - The UK’s Most Reliable Web Hosting Company Site (TopHosts.com) Netcraft names UK2.net top UK and European Web hosting company...
2. UK2 - The UK’s Most Reliable Web Hosting Company Site (TopHosts.com) Netcraft names UK2.net top UK and European Web hosting company...
3. UK2 - The UK’s Most Reliable Web Hosting Company Site (TopHosts.com) Netcraft names UK2.net top UK and European Web hosting company...
4. UK2 - The UK’s Most Reliable Web Hosting Company Site (TopHosts.com) Netcraft names UK2.net top UK and European Web hosting company...
5. UK2 - The UK’s Most Reliable Web Hosting Company Site (TopHosts.com) Netcraft names UK2.net top UK and European Web hosting company...

Welcome to website rankings help techniques (SEO) (the blog thats give you information on website rankings help techniques),

website-rankings help is a new talkware site helping you get the most from your website on the search engines (SEO) (website rankings help techniques). You will find out how to make changes to the website site that you own, or the website that you are producing to get the best website rankings help you can. We will provide useful tips to help you get the most from the internet so you can gain the best website rankings help your site can possibly get. Website rankings help techniques (SEO) is essential for you website as this is how people find you through search engines and marketing.

Search Engine Optimization (SEO) or website rankings help techniques, is an established process of improving how your web site interacts with search engines, thereby boosting your overall search engine rankings (website rankings help), visibility and popularity on the major engines. (SEO) search engine optimization (website ranking help techniques) involves many processes, including performing HTML code analysis, improving keyword density, enhancing content and page construction, testing, and verification.

This is exactly what website rankings do. And website rankings help do it so well, in fact, website rankings guarantee our performance that your website will work have better search engine rankings (website rankings help) (SEO) than it did before.

Website rankings help techniques (SEO) are critical for the profitability of your site. This is the most important thing in making your website being profitable is to get the website rankings help for your site as high as you possibly can. If you read the articles on this site about website rankings techniques help (SEO) you will find all the information that will help you get great website rankings help. If you would like help with website rankings help techniques (SEO)

The Web is filled with uncrawled web pages, and some of them may be yours.

One of the important first steps in optimizing a web site for search engines is to make sure that the pages of the site are search engine friendly, so that a crawling program from search engines can follow the links on pages to your site, and collect information from those pages. There are three major functions of search engines - crawling pages, indexing content, and displaying results to searchers - and the last two can’t happen until pages are crawled.

The purpose behind crawling web pages focuses upon:

• Discovering URLs for new pages,
• Acquiring content found on those newly discovered pages URLs, and;
• Finding Fresh content on previously crawled pages to keep search indexes fresh

As crawling programs explore the content on web pages, and come across URLs to new pages, they collect information about those URLs before actually visiting the pages they point towards. There may be a vast number of pages that a search engine has discovered a link to but hasn’t had a chance to crawl yet or may not be able to crawl for one reason or another - see Ranking the Web Frontier (pdf).

When a search engine discovers new URLs, it has to make some choices as to which pages to visit and index first. For example, if a search engine is faced with a choice between indexing a million pages of one site, or the home page of a million sites, it might choose to visit the million different sites first.

A recently published patent application from Yahoo attempts to provide a new approach to deciding which pages for crawling programs to visit based upon pages that it guesses might have the greatest search impact, addressing “Needy Queries.” Needy queries are search terms that people use with some level of frequency that have results that could stand being improved.

The patent begins by telling us about some past (and possibly present) approaches to deciding which pages to crawl and index content for based upon how important those pages might be perceived to be, rather than by a need for the content that the pages may contain, citing the following documents:

• Efficient Crawling Through URL Ordering
• Adaptive On-line Page Importance Computation
• Breadth-First Search Crawling Yields High-Quality Pages (pdf)

The fetching of content from URLs may be influenced by query independant aspects of those pages. If a URL has many links pointing to it or has accumlated a certain amount of PageRank, but hasn’t been visited and indexed yet, it might be moved up in a queue to be crawled, regardless of the content it might contain.

There is another approach to crawling URLs that focus upon content, know as focused crawling, in which crawling starts at pages that have been crawled and indexed before, and URLs are followed from those pages, on the assumption that those pages might link to other pages on the same topic. The patent points to another document that describes that kind of approach in more depth - Focused Crawling: A New Approach to Topic-Specific Web Resource Discovery (pdf).

It tells us that this kind of focused crawling focuses upon searching for pages that are relevant to a particular topic or a small set of topics, rather than by . Such focused crawling is guided by topic classification rather than the relevancy of queries issued by user requests.

Needy Queries

Are there needy queries, where searches are taking place that your pages might be good matches for that haven’t been crawled and indexed by search engines yet?

In Yahoo’s patent filing, the search engine doesn’t just consider information about the URLs that it might crawl to decide which pages to visit next, but also looks at its query logs to find queries that a number of people are searching for where the quality of the search results that show up in response to those queries may not be doing a good job of meeting searchers’ information needs.

The search engine would look at information that it might have about URLs that it knows about, but hasn’t visited yet to determine which pages to visit to try to improve the search results for those queries. That information could include words within the URL, the anchor text used for those links, how many links are pointing at that URL, and what the domain name is for the page the URL points at.

If that information seems like a good match for needy queries, those URLs may move up in the queue to be crawled sooner rather than later.

The patent filing is:

System and method for crawl ordering by search impact
Invented by Christopher Olston and Sandeep Pandey
Assigned to Yahoo.
US Patent Application 20090164425
Published June 25, 2009
Filed December 20, 2007

Abstract

An improved system and method for crawl ordering of a web crawler by impact upon search results of a search engine is provided. Content-independent features of uncrawled web pages may be obtained, and the impact of uncrawled web pages may be estimated for queries of a workload using the content-independent features.

The impact of uncrawled web pages may be estimated for queries by computing an expected impact score for uncrawled web pages that match needy queries. Query sketches may be created for a subset of the queries by computing an expected impact score for crawled web pages and uncrawled web pages matching the queries.

Web pages may then be selected to fetch using a combined query-based estimate and query-independent estimate of the impact of fetching the web pages on search query results.

This query-centric approach to determining an order for crawling web pages could lead towards:

Newer pages that might not have many links pointing to them showing up in search results quicker, and;

Long Tail queries that people are searching for with some frequency showing higher quality search results.

Web Slices are a cool new feature in Internet Explorer 8! With Web Slices, users can add little snippets of the web to the favorites bar and monitor their updates. Web Slices were introduced in a previous post here.  In this post, I want to walk you through creating a dynamic Web Slice in as little as 5 minutes!

What is a dynamic Web Slice?

Dynamic Web Slices use an Alternative Display Source (not to be confused with Alternative Update Source). As the name suggests, the content displayed in the preview actually comes from a ?live? web page which the Web Slice points to.  They are an addition based on feedback from the Beta 1 release of Internet Explorer and facilitate styling and cookie handling for authentication. A key advantage is that the web page is rendered in the preview window without losing any styling elements or active content. Hosting active content was not possible with static slices (Web Slices not using alternative display source). Static slices generate a preview of the sanitized entry-content element cached by the Windows RSS Platform which is stripped of script and other active content. Here is an example of a dynamic Web Slice from Live Search which has the display source pointing to a page on www.live.com rather than displaying cached content from the RSS store.

Web Slices using Alternative Display Source also make it easy for web developers to enable authentication within the preview window itself as well as have their customers protected against common internet security vulnerabilities such as Phishing and identity theft. Later this week we will be posting more details about Web Slice authentication. 

Moreover, a dynamic Web Slice is easy (and fun!) to create! So, set the clock for 5 minutes and let?s go right into making one!

Creating a ?live display? Web Slice

Let?s start with this basic template WebSlice.htm ?

<html>
<head>
<title>Web Slice Example</title>
</head>
<body>
    <div class="hslice" id="SliceID">
        <span class="entry-title">Slice Title</span>
        <a rel="entry-content" href="LivePreviewPage.htm" style="display:none;"></a>
        <span class="entry-content"> This content will appear on the page from where the user add?s the slice, but not in the preview window</span>
        <span class="ttl" style="display:none;">15</span>
        <a rel="bookmark" href="LivePreviewPage.htm" style="display:none;"></a>
    </div>
</body>
</html>

The class hslice helps Internet Explorer identify this snippet as a Web Slice. Fill in the template with the Web Slice id and title. Both are required properties. The title is displayed on the Favorites bar when the user adds the Web Slice.

The link tag?s entry-content attribute specifies the alternative display web page which is the source of the content displayed in the preview window of the Web Slice. This is where the RSS platform will look to see if anything has changed. The URL of the alternative display page is displayed in the navigation band at the bottom of the preview window.

The ttl property is optional and sets the default sync schedule for the Web Slice. This time can be modified to a higher value by the user. The user can hit the refresh button on the navigation band to refresh the alternative display web page within the preview window.

The optional bookmark property can be used to navigate to the alternative display web page (or any other page) when the user clicks on the Go arrow on the navigation band. The page is then opened in the current tab in the browser.

That?s it! You?re done! Wasn?t that super easy?

Serving ?live? content

A few key things to note here -

As per the sync schedule, the Windows RSS Platform will ping WebSlice.htm, not LivePreviewPage.htm (the display source).

1. If there is any change in WebSlice.htm, a fresh copy of WebSlice.htm is fetched. When the user clicks on the Web Slice, the current copy of LivePreviewPage.htm is served and cached.
2. If there is no change to the title and entry-content property of WebSlice.htm, when the user clicks on the Web Slice, the previously cached copy of LivePreviewPage.htm is served. In this case, there could be inconsistency between the preview window display and the actual web page.

In both cases, if the user manually hits refresh, the current copy of LivePreviewPage.htm is displayed and cached. Thus, here are a few tips in order to ensure that the content in the preview window is ?live? -

• Update WebSlice.htm for changes to reflect in accordance with the set sync schedule.
• If you want to make sure that the user always sees a ?live? copy of LivePreviewPage.htm every time she clicks on the Web Slice add the no-cache Cache Control header to LivePreviewPage.htm. This will ensure that the display in the preview window is always consistent with the actual LivePreviewPage.htm.
• You can also change the title on WebSlice.htm to update with the sync schedule. This is especially useful if you want to display the updated temperature in the title bar for a weather Web Slice.
• Another optional property you can use is endtime. This specifies the expiration time of a Web Slice. For example, it can be used to indicate an expired item for an auction Web Slice.

Design a usable dynamic Web Slice

Dynamic Web Slices retain all the styles from the preview web page, including those inherited from external stylesheets and look just like the actual web pages. However, there are a few things to bear in mind ?

1. The default size for dynamic Web Slices is 320x240. We strongly recommend that you design your Web Slices to conform to this size. Users do have the option of manually resizing a Web Slice for their convenience, but we wouldn?t like to compel them to do so in order to see all your content.
2. Another consideration to bear in mind is performance. We encourage you to leverage the functionality available with dynamic Web Slices, but to make sure that it renders in less than 500ms to keep the preview window usable.
3. While users are able to navigate within the preview window itself, it is not encouraged to host navigations to complex pages. The preview window has stricter security restrictions and blocks dialogs, the Information bar, popups etc.  Moreover, Web Slices are a great way to attract users to your actual web site and you can explicitly set links in preview windows to open in a new tab in the browser by using the target="_blank" attribute. Note that cross zone navigations from the preview window are blocked.

<a href="http://www.example.com" target="_blank">This link will open in the new tab in the browser</a>

The Web Slice Style Guide has a section on Web Slices using Alternative Display Source containing great tips to make your Web Slices look pretty!

You can see how simple and useful they can be! I hope that you will have fun creating really neat Web Slices for your web pages. The IE 8 Gallery has a ton of great Web Slices along with other IE add-ons.

Ritika Virmani
Program Manager
Internet Explorer  - Web Slices and Navigation

Web Deployment has taken a huge stride in Visual Studio 2010.   I have started a blog series where I have written about web deployment, you can read more about them below:

This is how you can use web packages on any machine with MSDeploy and by configuring your deployment options in the destination manifest you can go and and easily recreate your webs.

It is not possible for someone to come up with every possible provider that everyone needs so there will be an extensibility model by which you can write your own providers and register it with MSDeploy engine.

Visual Studio is also made extensible to allow you to hook into the packaging and publishing process to call your custom MSDeploy providers in the source manifest.

The most interesting pieces is that with IIS Manager and Visual Studio 2010 UI, you will not really need to know all these details, things will just work but I thought it is often interesting to know how things work behind the scenes.

I hope this conceptual overview helps you get the perspective on how web deployment with VS 2010 and MSDeploy will work!!

Vishal R. Joshi | Program Manager | Visual Studio Web Developer

1. Web Sites Disrupted By Attack on Register.com (Washington Post) Web site host and domain name registrar Register.com has been...
2. Web Sites Disrupted By Attack on Register.com (Washington Post) Web site host and domain name registrar Register.com has been...
3. Web Hosting Leader Go Daddy Wins “Best of the Best” Award (TopHosts.com) Web Host and Domain Registrar Honored as Arizonas Favorite Technology...
4. Google angles for a place on pristine Web sites (CNET) The search company is angling to have Web hosting companies...
5. Rambosk campaign Web site disappears after allegations raised by former staffer (Naples Daily News) Call it the case of the disappearing Web site. On...

BizIII - A Daily Podcast with
Ideas
Information
Inspiration
to help you make more money ...

with Les Bain and Mike Hughes

Share This Show with a Friend

Send a Story Idea to BizIII

So I was visiting with a friend last night, and he indicated that he was having a bit of a problem with his WSS 3.0 installation.  Short story is that he has a dedicated Win2k8 box acting as a web server on his domain for internal sites.  They have several web-based LOB apps that run on that box, all as virtual directories under the default web site.  Even though they are running SBS 2003 with its WSS 2.0 companyweb, they wanted to install WSS 3 to take advantage of the new wiki site template.  So, they installed WSS 3 on the web server, which immediately broke their LOB apps.

So what happened?

When you first install WSS 3.0 and run the SharePoint Configuration Wizard, SharePoint creates a new web application (SharePoint – 80) and creates a new web site in IIS that takes over the default site.  Dana recognized this, so within IIS he edited the bindings for the SharePoint site to use port 81, allowing him to re-enable the original default website in IIS and get his LOB apps back.  The problem?  Not only was it a pain having to enter :81 after the servername to access the site, but clicking links on the SharePoint site continued to want to use port 80, resulting in constant 404 errors.

So how did we fix it?

If you’re new to SharePoint, it is worth taking a little time to explain some of the architecture and terminology around SharePoint 3.0 to help put the answer in to context.  First, it is important to understand the distinction between a SharePoint web application, and an IIS web site.  SharePoint (whether WSS or MOSS) can have multiple web applications.  These are created via SharePoint Central Administration.  You can think of a SharePoint Web Application as your top-level SharePoint site – but it is distinctly different from a website in IIS.  An IIS site that is mapped to a SharePoint web application can be thought of as a gateway to access the SharePoint web application.  You can delete the site from IIS without affecting any of the content in the SharePoint application.  (Obviously you won’t be able to access the SharePoint web application without an IIS site, but none of the SharePoint web application content or configuration is stored in the IIS site). 

There are several benefits to this approach – including the ability to have multiple IIS sites mapped to a single web application, with each site being bound by a different SharePoint security zone.  The distinction between the web application and the IIS site in Dana’s situation is that the original IIS site that was bound to port 80 with no host header was separate from the actual SharePoint web application, and even though that was the initial IIS site created to access the SharePoint web application, it isn’t necessary to use that IIS site.

The simplest solution for Dana was to create a new IIS site that used a host header to access his SharePoint web application.  This is actually very simple and straight-forward to do from within SharePoint Central Administration:

1. Open SharePoint Central Administration  (Start | Administrative Tools | SharePoint 3.0 Central Administration) on your SharePoint server.
2. Click on the Application Management tab
3. Click on the link to Create or Extend an Existing Web Application
4. Click the link to Extend an Existing Web Application  (we are extending an existing web application to another IIS site)
5. Select the web application you want to extend.  (The default SharePoint web application on a stand-alone WSS installation is SharePoint – 80.  On SBS 2008, the companyweb application is  remote.yourdomain.com:987  )
6. Select the option to create a new website and enter a description that is meaningful to you  (this will display in IIS)
7. Change the port to 80
8. Enter a value for the host header  (in Dana’s case, we used   wiki  - obviously, you will need to create the necessary DNS records so your host header name can be resolved via your internal DNS.  I personally prefer to create a CNAME (alias) that resolves to the host (server) that is running SharePoint.  Alternately, you could also create a new A record).
9. In a typical small business deployment, you will accept the default security configuration options
10. Select the appropriate zone and click OK.

This will create a new site in IIS that is mapped to the web application you selected.  After we had created the new site for Dana and created the necessary CNAME record for  wiki  in his DNS, we were able to browse to http://wiki on his internal systems and access the SharePoint application successfully, and navigate without 404 errors.

Additionally, we were able to delete the original IIS site that Dana had changed the bindings to port 81.  Since Dana & co were now accessing the web application via the new site (http://wiki) we didn’t need the original site on port 81 any more.  We also did this within SharePoint central administration:

1. Go to the Application Management tab
2. Click the link to Remove SharePoint from IIS Web Site
3. Select the web application
4. Select the site
5. Optionally select to delete the site from IIS  (which we did select in Dana’s case)

So why was Dana getting the 404 errors after he changed the bindings to a new port number in IIS?  If you go back to the page where we extended the web application, take note of the description under the Load Balanced URL section:

The description reads:  “The load balanced URL is the domain name for all sites users will access in this SharePoint Web application.  This URL domain will be used in all links shown on pages within the web application.  By default, it is set to the current servername and port.”

When the SharePoint Configuration Wizard created the initial web site in IIS, the SharePoint load balanced URL for that site was http://servername:80  -  which will resolve to the default website on that server.  When Dana changed the port to 81 and re-enabled the original default website, links in the SharePoint web application (when accessed from the original IIS site) all used the Load Balanced URL, which resolved to the re-enabled default website on port 80 – thus resulting in the 404 errors.

The moral of the story here?  Well there are a couple:

• You can have as many IIS sites linked to a single SharePoint web application as you want.
• When administering SharePoint, do as much as you can in SharePoint Central Administration.  Chances are you won’t get the results you want if you try to make changes (such as site bindings) via IIS.

One of my personal rules when it comes to IIS is to leave the default website alone.  Personally, I always create new websites in IIS and use host headers to access those sites – so everything is accessible on port 80 (assuming http) and users don’t have to remember weird port numbers, etc.  Additionally, using host headers gives you the freedom to move websites to different web servers without affecting the end-user experience.  Just update your DNS record for the host header value to point to the new server and voila! – users are accessing the same site via the same URL and have no idea it has been moved to a different physical box.  And this is true of all web applications I use – DotNetNuke, Community Server, etc. 

The only exception to my rule of putting each web application in their own IIS web site is when we need multiple apps all on the same server accessible via SSL.  Since SSL traffic is encrypted, IIS is unable to inspect the host headers, meaning it can only direct SSL requests to the correct site based on the IP / port combination.  So, to have multiple web apps on a single box accessible via SSL, we either need to have multiple sites all on one IP listening on different ports (443, 444, etc.), or multiple IPs on the box so each site can listen on 443 on a separate IP, OR configure the different web applications as virtual directories under one IIS site that is listening on 443 for the one / all IP addresses.  Depending on the number of applications you need accessible via SSL, it can makes more sense to configure those apps as virtual directories under a single site, so you reduce your administrative overhead by not having to administer multiple IP addresses / ports / SSL certificates.   But even then, I create a new site in IIS to put everything under instead of using the default site.  Yeah, I know – I’m weird like that 

Of course – there is always more than one way to skin a cat, so there is a completely different method we could have taken to fix Dana’s issue as well.

Let’s say there was a business need for Dana’s web applications (that were all virtual directories under the default site) to be accessible as virtual directories under his SharePoint site.   This approach was actually recommended to Dana by other individuals telling him to add an Application Exclusion to the SharePoint site.  Dana couldn’t find out how to do this – but there is good reason why:  Application Exclusions don’t exist in SharePoint 3.0.

Here’s the deal:  SharePoint 2.0 and 3.0 have considerable distinctions in their architecture.  For example, when you extended SharePoint 2.0 to a website in IIS, SharePoint assumed that the entire IIS site would be devoted to the SharePoint application.  As a result, if you wanted to have non-SharePoint virtual directories under the IIS site, you had to tell SharePoint 2.0 to exclude those virtual directories from its management, allowing the web applications in those virtual directories to work as intended. 

SharePoint 3.0 uses a different approach.  Instead of assuming the entire IIS site is devoted to the SharePoint web application, you have to explicitly tell SharePoint what paths in the IIS site are managed by SharePoint.  When we create a new SharePoint Web Application, SharePoint assumes that it will manage the root path as well as everything below the /sites/ path.  (Hint: when you create a new web application and are on the Create Site Collection page, this is why you have the to options for the URL:  http://hostheader/  or http://hostheader/sites/ )

What this means is that SharePoint 3.0 plays very nicely with other web applications in the same IIS site.  So in Dana’s case, when he first installed SharePoint 3.0 and it created a new IIS site that replaced his original default website, he could have simply recreated the virtual directories for each of his web based LOB apps under the IIS site SharePoint created as long as none of them used the sites name, since that was defined as a Managed Path for the SharePoint web application.  And even then, if he wanted to use the sites path for a non-SharePoint application instead, he could have removed the sites path from SharePoint management.

You can administer SharePoint’s managed paths from SharePoint Central Administration.  Simply navigate to the Application Management tab and click the link for Define Managed Paths.  When you add a managed path, you specify what type of inclusion it will be.  There are two types of inclusions – an explicit inclusion and a wildcard inclusion.  An explicit inclusion means that SharePoint will manage just that path, where as a wildcard inclusion tells SharePoint that every path under the wildcard inclusion path should be managed.  This is particularly useful if you are enabling self-site creation for users, so they could effectively create their own site collections (top-level SharePoint site) under a common directory (e.g /sites/).