• The WSS 3.0/SharePoint 2007 WYSIWYG editor can’t upload images, video, audio clips and can’t insert Flash files.
  The workaround requires so many mouse clicks and web page interactions, that users look at me like I have “3 heads” when I show it to them.
• Working with tables is difficult, at best
• Any customizations such as:
  • custom file browsers,
  • link management,
  • syntax aware code viewers,
  • and much more

  are difficult to implement

From a Wiki perspective

• Explicit syntax [[]]
• No free form image inserts
• No server side include of other topics.
  Need an easy way to include content from another site. In Foswiki/Twki the %{link} tag can be used
• No easy upload and insert of attachments
• No Easy way to insert links.
  Foswiki/TWiki support WikiWord/Camel Case.
• No tagging
• WYSIWYG support for tables is ugly. You must know the number of rows and columns in advance.
  Foswiki/TWiki support simple “|” character based table definitions and an editable table plugin
• No printable vew
• No export to PDF/Word
• Limited Alerts support.
  Email alerts say something changed and include the full text. However, this can not be customized to tell you what changed. You must login and see the history for that.
  Foswiki/Twiki support a highly customizable WebNotify
• No easy support for multiple top level webs.
  There is only one flat “Wiki Pages” page with list view, no hierarchical view or some sort of sitemap feature.Wikis are made up of Topics that are interlinked in the form of a graph. Hence a specific Tree or Hierarchy structure is not the right representation. However, there must be a way to find all top level topics, i.e. Topics that are not linked to by any other topics. These topics will include
  • the starting points
  • new topics that have been added but not yet organized.

  I could not find a way to do this with this with the SharePoint Wiki. Even a simple way to flag a page as “add this page” to “Quick Launch” would help.

• Not easy to provide a standard footer
  No apparent concept of templates

The “MS/SharePoint 2007″ update approach does not seem to fit into this “revolution”, either. The current approach requires direct access to the servers to install and activate extensions. With “Cloud Computing” and web hosting on the rise; the direct server access approach is not a good match. SharePoint needs something like a Plug-in central for SharePoint. Ideally, this could work on a site level and not require Central Administration be updated first. Another issue for remote management is that “by default” SharePoint Central Administration and SharePoint Sites must run on different tcp/ip ports. I’m sure that if I work real hard I can find a way to run them on the same port using “host-headers”. However, if I do, then will this really be supported or would the next next patch or upgrade break my sites?

So my wish list for SharePoint 2010 starts with:

• A site focused web based upload, install, and management capability similar to that found in WordPress and Plug-in central for WordPress
• A Rich-Text Editor that is rich in function, easily expandable and replaceable.
  Consider integrating a freely available javascript-based Rich-Text Editor for all browsers including IE. My primary choices would be FCKEditor and TinyMCE or TinyMCE Advanved.
• A competitive wiki and blog, without having to buying 3rd party add-ons.

Perhaps Microsoft only wants SharePoint to be a document sharing facility. Maybe I should not be trying to bend WSS and SharePoint into the social media and web 2.0 “arena”. Maybe that “arena” should just belong to Apache, Linux and Open Source.

The question is:“How can we get all of these PDFs reprocessed to be searchable for a reasonable cost?” I spent some time reviewing various PDF tools and was surprised that few were “truly” scriptable in batch. Some tools would not run on a server or could not interface with a “HTTP Network Place”. There were some development kits that were marketed and priced for you to build your own PDF tools and they have a pretty spendy price tag. These were out of scope with this year’s budget. Some, like Acrobat, had a batching mechanism in their GUI. However, this would require us to:

• Manually checkout all of the PDF’s
• Download them into a folder structure
• Batch them in the UI
• Upload all of the OCR’ed PDFs
• Manually check-in all of the PDF’s

I don’t think so! We are talking about tens of thousands of documents. This would be labor intensive and prone to human error.

I’d just about given up finding something that would fit into the current budget, when I received an email notice about Solid Converter PDF V5 being available. While reviewing the v5 features, I noticed Solid PDF Tools v5 was out and it was just $20 more the upgrade to Solid Converter PDF v5. Low and behold, Solid PDF Tools v5 claims that it has a scriptable interface. I got to looking further and discovered their script reference manual and developer blog. I downloaded the trial and verified it. The software worked, I sent them my $59 upgrade fee and viola, I had a “scriptable enough” tool that I could re-scan “image-only” pdf files to create “searchable” pdf files. I say “scriptable enough” because it gets the job done, but Solid PDF Tools v5 needs to load the GUI, load the splash screen, and display the UI for each file processed. This seems to add 10-15 seconds to the processing time of each file

Now to convert the “image-only” PDF’s in SharePoint. Once again, I decided to try PowerShell rather than write a C# program to interface with SharePoint. At the same time, I decided to give the PowerGUI Tools a try. I found the PowerGUI Script Editor to be quite useful for developing, debugging and running my script.

The “proof-of-concept” result is ~100 lines of PowerShell code that

• processes all of the webs in a sharepoint site
• processes all of the folders and sub-folders
• processes all of the PDF documents and sends them to OCR processing

## References
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint") 
[void][System.Reflection.Assembly]::LoadWithPartialName("System.IO") 
 
$SolidPDFTools = "$env:SolidPDFTools.exe" 
$LocalFileFolder= "D:\spwork\input";
$OCRWorkFolder= "D:\spwork\output";
$OCRWorkLogFolder= "D:\spwork\logs";
$OCRScriptFile= "D:\spwork\ocr.sdscript";
 
function script:write_local_file($file, $fileFolder) {
	$fs = New-Object System.IO.FileStream($(Join-Path $fileFolder $file.Name), [System.IO.FileMode]::Create)
	$bw = New-Object System.IO.BinaryWriter($fs);
	[Byte[]] $binfile = $file.OpenBinary();    
	$bw.Write($binfile);
	$bw.Close();
	$fs.close();
}
 
function script:ocr_local_file($file, $in_fpath, $out_fpath) {
	$infile = Join-Path $in_fpath $file.Name;
	$infile = $infile -replace("\\","\\");
	$outfile = Join-Path $out_fpath $file.Name;
	$outfile = $outfile -replace("\\","\\");
        $logfile = Join-Path $OCRWorkLogFolder "debug.log";
	$logfile = $logfile -replace("\\","\\");
	$DBG='<</Level/Verbose /Emit (starting) /FileName (' +  $logfile +') >> Trace';
	$INP='<</Input (' + $infile + ')';
	$OTP='/Output (' + $outfile + ')';
	$OCR='/OCR/Searchable';
	$CRE='>> Create';
	$XIT='EXIT';
	Write-output $DBG | Out-File $OCRScriptFile -encoding ascii;
	Write-output $INP | Out-File $OCRScriptFile -append -encoding ascii;
	Write-output $OTP | Out-File $OCRScriptFile -append -encoding ascii;
	Write-output $OCR | Out-File $OCRScriptFile -append -encoding ascii;
	Write-output $CRE | Out-File $OCRScriptFile -append -encoding ascii;
	Write-output $XIT | Out-File $OCRScriptFile -append -encoding ascii;  
 
	$sTemp = &SolidPDFTools /i $OCRScriptFile /f script 
	Write-Host $sTemp;
}
 
function script:upload_ocr_result($file, $ocr_fpath) {
	$ocrfile = Join-Path $ocr_fpath $file.Name;
	$fs = New-Object System.IO.FileStream($(Join-Path $ocr_fpath $file.Name), [System.IO.FileMode]::Open)
	$br = New-Object System.IO.BinaryReader($fs);
	[Byte[]] $binfile = $br.ReadBytes($br.BaseStream.Length);
	$file.SaveBinary($binfile);
	$br.close();
	$fs.close();
 }
 
function script:process_a_folder($folder) {
	$files = $folder.Files;
	foreach($file in $files) {
		if ($file.Name.ToLower().Contains(".pdf") ) {
			if ($file.CheckOutStatus -eq "None") {
				$file.CheckOut();
            			write_local_file $file $LocalFileFolder;
				ocr_local_file $file $LocalFileFolder $OCRWorkFolder;
				upload_ocr_result $file $OCRWorkFolder; 
				$file.CheckIn("version has had OCR processing performed");
				$file.Update();
				}
			}
		}
	$sub_folders = $folder.SubFolders;
	process_folders($sub_folders);
}
 
function script:process_folders($folders) {
	foreach($folder in $folders) {
		process_a_folder($folder);
	}
}
 
function script:append-path {
	$oldPath = get-content Env:\Path;
	$newPath = $oldPath + ";" + $args;
	set-content Env:\Path $newPath;
}
 
# MAIN
append-path (resolve-path 'D:\Program Files\SolidDocuments\Solid PDF Tools\SPDFT').Path
 
$site = new-object Microsoft.SharePoint.SPSite("http://my-sp-server");   
$siteweb = $site.OpenWeb();   
$webs = $siteweb.Webs;   
foreach($web in $webs) {   
	$folders = $web.Folders;
	process_folders($folders);
	}

I do plan to add some logging, change the hard-coded variables, and look at using streams instead of Byte[] to be more flexible and scaleable. I’ll need some error handling to deal with things like download or upload failures before I can run this in production. I’m also trying to determine if a PDF is “image only” or searchable. However, the “Proof-of-concept” does work.

stsadm -o activatefeature -name feature-name -url websiteurl -force

The synopsis of the code is:

• Start with the sharepoint site object.
• Loop through all of the sub-sites.
  • Get the website url for the sub-site.
  • Call stsadm to activate the feature for the sub-site.

The PowerShell code

## Reference to SharePoint DLL
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")
## Probable location of sharepoint STSADM utility program
$stsadm = "$env:programfiles\Common Files\Microsoft Shared\Web Server Extensions\12\BIN\STSADM.EXE" 
 
############################################
# Activate-feature-onSite [-feature  |-url  ]
############################################
function global:Activate-feature-onSite($feature, $url)
{
$spsite=new-object Microsoft.SharePoint.SPSite($url); 
 
for($i=0; $i -lt $spsite.AllWebs.Count;$i++){
   $websiteurl=  $spsite.AllWebs[$i].url;
   $sResult = &stsadm -o activatefeature -name $feature -url $websiteurl -force
   if(!($sResult -like "*Operation completed successfully*")){
      Write-Host -ForegroundColor "red" -BackgroundColor "white" "Activate of feature '$feature' for '$websiteurl' Failed! `n $sResult"
     }
   }
$spsite.Dispose();
} 
 
# to call
Activate-feature-onSite "feature" "http://site"

• This first way, involves placing an HTML file on your site using an Google generated HTML file name.
  This must be placed in the root directory of you site.
• The 2^nd way involves placing a Google generated meta tag in your site’s default webpage.

Based upon my previous experience with SharePoint, I tried the HTML file first. Using Internet Services Manager I determined the physical directory that was mapped to the virtual directory for my SharePoint site. Next, I created an HTML file in that directory using the name that Google provided and “viola”, I figured that I was done. Unfortunately, the way SharePoint sites are registered into IIS, a SharePoint site would not render my HTML file from this location. This results in a 404 error. On to the meta tag option.

My 1^st attempt at the meta tag option was to create a hidden “Content Editor Webpart” on my site’s home page and place the meta code in it. I suspected that this would not work as the meta tag needs to go into the head section of the HTML page before the body section. Sure enough, it did not work.

My next approach was to determine which file that I could modify to place my meta tag in.
I’m sure that there is a SharePoint Designer way to do this. However, if you are using the free version of SharePoint 2007, then I figure you may not want to buy SharePoint Designer.

SharePoint sites open with a default.aspx file that is based on the site type. There is are seperate default.aspx for Team Sites, Blog Sites, etc. Each of these default.aspx files include a default.master. The default master can be overridden on a site by site basis by editing the default.master in your site’s “Master Page Gallery” by accessing “Site Actions” –> “Site Settings” and then selecting the “Master Pages” Link in the “Galleries” section. Rather than change the default.aspx which would affect all sites of a type running on my server, I decided to change the default master for my site.

To change the default.master, you will:

• check it out.
• download a copy
• edit it with notepad (or some other text editor) and place your meta tag
  I chose to place my Google meta tag just below the
  <META HTTP-EQUIV=”Expires” content=”0″>
• upload the update
• check-in the file

You can verify that this is correct by visiting your site’s homepage and then viewing the page source via your Web Browser. If you can see your verify code, you are ready to verify your site with Google and move to publishing your sitemap.

Getting my sitemap into google
In order to get Google to download my sitemap, the sitemap file, sitemap.xml in my case, would need to be anonymously accessible form a url on your site that Google lets you specify. I already knew that I could not simply place the sitemap file into the root directory of my website. My solution was to create a SharePoint document library in my site, provide anonymous read access and upload the sitemap.xml file to that library.
Then I entered the sitemap url into Google.

WSS 3.0 provides a control SPSecurityTrimmedControl that allows the conditional rendering of content based on a users permission. Using this control, I was able to restrict who would be shown People and Groups on my site.  To do this I modified the people.aspx and the groups.aspx files for my site.

A word of caution: I have no doubt that when applying a future service pack, Microsoft will overwrite my changes. Therefore I’m managing my changes externally from the Microsoft SharePoint directory. This will allow me to difference new code from Microsoft and reapply my changes.

people.aspx
The file people.aspx resides in the \Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS folder. It lists the persons in the site. After I made a backup copy, I opened the file in a text editor and located the start of the “PlaceHolderMain” asp:Content section and added the start of my permission change

before

<asp:Content contentplaceholderid="PlaceHolderMain" runat="server">
	<asp:PlaceHolder id="PanelAllPeople" runat="server">
	 <div class="ms-listdescription">

after

<asp:Content contentplaceholderid="PlaceHolderMain" runat="server">
<SharePoint:SPSecurityTrimmedControl runat="server" PermissionsString="ManagePermissions">
	<asp:PlaceHolder id="PanelAllPeople" runat="server">
	 <div class="ms-listdescription">

I located the end of the “PlaceHolderMain” asp:Content section and completed my permission change.

before

	<asp:HyperLink id="LnkCallTo" runat="server"
		Text="<%$SPHtmlEncodedResources:wss,people_callbuttontext%>"
		style="visibility:hidden;display:none"
		NavigateUrl="callto:" />
</asp:Content>

after

	<asp:HyperLink id="LnkCallTo" runat="server"
		Text="<%$SPHtmlEncodedResources:wss,people_callbuttontext%>"
		style="visibility:hidden;display:none"
		NavigateUrl="callto:" />
</SharePoint:SPSecurityTrimmedControl>
</asp:Content>

I also located the modifed the “PlaceHolderLeftNavBar” asp:Content and made the similar changes as above to require the user to have ManagePermissions as well.

groups.aspx
The file group.aspx resides in the \Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS folder. It lists the groups in the site. After I made a backup copy, I opened the file in a text editor and made the same “PlaceHolderMain” asp:Content section changes as I had to people.aspx.

Overview of SPSecurityTrimmedControl

The SPSecurityTrimmedControl conditionally renders the included contents only when the current user has the permissions defined by the PemissionsString.
An example requiring the user to have “ManagePermissions”:

<Sharepoint:SPSecurityTrimmedControl runat="server" PermissionsString="ManagePermissions">
<!--
...
... code being managed ...
...
-->
</SharePoint:SPSecurityTrimmedControl>

Here are some of the values for the PermissionsString

The full list is on msdn.