Checklist for Designing and Implementing SharePoint 2010 Extranets – High Level Items to Consider

I have been designing SharePoint extranets since MOSS 2007 days and it’s been amazing to see that even though on surface each extranet projects are approached same way, each and every extranet projects provides different architectural challenges. Recently I have attended Jeremy Thake’s webinar on what items needs to consider while designing extranet systems – Governing your Extranet for a better user experience and I was surprised to learn many new facet of SharePoint 2010 extranet design.

His webinar motivated to write detailed article on my experience and high level items needs to be considered while designing and implementing SharePoint 2010 extranets. Hopefully this article would provide general checklist & guidelines require to design SharePoint 2010 extranets.

Understand Extranet Type based on Business Requirements & Usage Scenarios

• Define the User Personas - Employees, Partners, Vendors/Customers
• Externally Available Intranets or Collaborative Sites for Employees without requiring logging into VPN or Corporate Network – Extranets for Remote Employees
• Typically extranets are platform shared with external users such as partners, vendors, and customers
  • Shared Collaborative environment with Partners or Customers – External facing Team Sites (e.g. Customer Portal, Partner Portal)
  • Internet facing read only documents, wiki sites, or shared collaboration environment – Publishing Feature (e.g. Marketing Sites, School Portal, Blogs & Discussion Forums)

Typical Extranet Project/Implementation Team

• Part-time involvement from IT Teams – Infrastructure Team, Security Team, Network Team
• Ideal Full-time Project Team – Product Owner, Business Analyst/Project Manager, SharePoint Architect, SharePoint Administrator, More than 1 SharePoint Developer, SharePoint Quality Assurance, User Experience Architect

Infrastructure Considerations

• Core SharePoint Infrastructure & Network Topology – UAG, Firewalls, DMZ, Servers, Network, DNS, Databases, SAN
• Extranet Network Topology
  • Typically decides where would be SharePoint Servers located – In corporate network or DMZ
  • Typically decides high level SharePoint Server Topology and SharePoint Architecture
  • Topologies to Consider – Edge Firewall Topology, Back-to-Back Firewall Topology, or Split Back-to-Back Firewall Topology – e.g. Configure UAG in DMG to protect extranet farm hosted in corporate farm using Edge Firewall Topology
• Server and Farm Topology
  • Single Farm vs. Multiple Farms
    • Do you really require separate farm? – Impact on licensing, hardware, security, physical data separation etc.
    • Options are Single farm with same sites serving both intranet and extranet (e.g. Same Web Application serving both intranet/extranet in Single Farm), different sites for intranet or extranet environment (e.g. Multiple Web Applications in serving both intranet/extranet in Single Farm), or Multiple farms for physical separation (e.g. Multiple Web Applications serving intranet and extranet in different Farm)
  • SharePoint Farm Architecture – Web Front Ends, App Servers, DB Servers
    • Hardware vs. Software Load Balancer for Web Front Ends
    • Install SSL certificate on SharePoint web application
  • Cross-Farm Infrastructure for Multiple Farms
    • Shared SharePoint Services – User Profiles Service, Search Service, Managed Metadata Service etc.
  • Virtualization, High Availability,  Backup-Restore Approach, and Disaster Recovery
  • Global Availability and Latency – WAN Acceleration with Central Farm vs. Global Farms in Multiple Locations with Data/Documents Replications

Security and Identity Management Considerations

• Identity Management System
  • Internal and External Accounts should be in separate identity management system.
  • Understand Types of Users
    • Internal Users – in most cases, it’s AD
    • Extranet System Managed Users – AD, ADLDS, SQL, LADAP
    • Extranet System Federated Users – ADFS
    • Extranet System Open ID or Social System Users – Live ID, Google, Facebook, Twitter, LinkedIn etc.
  • Sample Configurations
    • Single AD with Same OU or Multiple OU for both Internal and External Accounts – Windows Authentication is sufficient
    • Multiple AD with Two-way Trust for both internal and External Accounts – Windows Authentication is sufficient
    • Multiple AD with Single-way Trust for both internal and External Accounts  – Requires Claims & LDAP FBA
    • AD for Internal Users and ADLDS FBA for external Accounts  -  Requires Claims based authentication
    • AD for Internal Users and SQL/ASP.NET FBA for external Accounts  – Requires Claims based authentication
    • AD for Internal Users and ADFS (Web based SSO federation) for external Accounts – Requires Claims based authentication
    • AD for Internal Users and Windows Live ID for external Accounts – Requires Claims based authentication
• Authentication – Account/Identity Management
  • It is important to note that SharePoint doesn’t perform Authentication
  • Decide whether to use Classic (Windows – NTLM or Kerberos with Internal AD) or Claims (LDAP/SQL/FBA/ADFS/ADLDS etc.) based Authentication.
  • It is important to note that regardless of what Authentication Source or Authentication Type is, SharePoint treats all users as SPUser object. SPUser object would contain user token based on authentication type or authentication source.
  • Does Kerberos need to be  enabled to pass credentials to the internal systems? Claims are built to avoid Kerberos delegation to pass Claims without concerns of multiple-hops.
• Login Experience
  • Classics Authentication – Mixed-Mode Authentication - MOSS 2007 way
    • When to use? Different protocols like HTTP or HTTPS for internal vs. external users, separate environments or URLs for internal and external users, Single Sign on for internal users in corporate network
  • Claims Authentication – Multi-Mode Authentication – New in SharePoint 2010, Provides option to Choose Authentication Type before Login Prompt
    • When to use? Single URL for both internal & external users (There is exception – if both internal & external users are in same AD or multiple AD with two-way trust with windows authentication can have single URL), Must be used for Live ID, Must be used to federate between two organizations,
  • Custom Login Page
    • Most customer facing application requires custom branded login page. Requires custom development for branded login page. Out of box login options may not be sufficient for externally facing portals.
    • Optionally use Third-Party SharePoint Protection & Reverse-Proxy lookup Tools like UAG as long as these tools supports authenticate logic for all configured identity management systems.
• Authorization – Site Membership
  • Unlike Authentication, SharePoint performs Authorization by assigning SPUser object to SharePoint Security Groups
  • Two Kind of Authorizations driven by Site Taxonomy
    • Shared Sites/Pages like Yahoo and Dedicated Sites for customers
    • Driven by Customer SLA & Sites Hierarchy – Separate/Dedicated Site Collection for each Customer Site or Single/Shared Site Collection for Multiple Customers
  • Protecting Content
    • Driven by User Personas, User Types, and Site Hierarchy
    • Site Level Permissions Inheritance – Inherit Security or Break Security
    • Site Security Groups -Use Out of box Security Groups or Create New Security Groups based on Out of box Permissions
  • Site Membership
    • Consider Automated Security Group and Site Membership Provisioning and Cleanup Process
    • Either Assign Users or Groups to the SharePoint Security Groups
      • Assign AD or ADLDS Groups to the SharePoint Site Security Groups, AD Groups are recommended for account maintenance if users are in AD. Map these AD groups to SharePoint Security Group for ease of Site Membership management
      • Assign individual users to the SharePoint Site Security Groups – This may require for ADFS
    • Define process to delegate site membership, Make business users/site owners to manage site membership
    • If external users or customers are managing site membership, Use People Picker filtering mechanism to restrict external users visibility in internal directories. Use stsadm -Peoplepicker-searchadcustomquery for AD. Implement custom filtering in Find methods of FBA/ASP.NET Membership Providers
• User Life Cycle Process
  • In most cases, extranet environments are controlled environment which doesn’t require user registration process. User registration typically requires for public facing internet sites.
  • Needs to define process for User Provisioning & Decommissioning
    • Define business process to request provisioning new users – both in bulk & individual
    • Define needs for Shared User Accounts or Dedicated User Account
    • Consider Auto User Provisioning Process  and Decommissioning Process
  • Self-Service User Management – Needs to define self-service or IT managed User management process - how user would reset their passwords, how users would request access to the sites, how users would be given access to the sites etc.
  • User Monitoring & Auditing – It’s a process challenge, external users not sneaking in from back door – Proper User Validation, Expiration, and De-Provisioning  (e.g. Verify users once a 3 months), Either build custom tools or use third-party ISV products for Identity Management

Information Architecture Considerations

• Logical Architecture, Site Hierarchy, Site Taxonomy
  • Web Application, DNS, Host Header, and Application Pool
    • Single or Multiple  SharePoint Web Apps
    • When would you require Single SharePoint Application? – Single URL
    • When would you require Separate SharePoint Application? – different URLs or Authentication Settings
  • Site Collection vs. Sites – Extranet Sites Hierarchy and Number of Sites based on Taxonomy
    • In most cases, SLAs, Security Isolation & Data Protection drives this design. Use Site Collection if Security is boundary and users will have full control. If dedicated content database is important, use site collection as well.
    • Use Sites for Shared Access scenario where multiple customers will have read-only access to the content or contribute access to shared data. As long as customers can’t manage security, you are OK having this model.
    • Plan to use dedicated Site Collection for customer/partner centric portals. You can use SharePoint Multi-Tenancy framework as well for host named site collections. This is how Office 365 or Hosted/Cloud environments work.
  • Single Site to serve All Customers or Dedicated Sites for Each Customer
    • Review business requirements to see if there are needs for dedicated collaborative environments like document libraries, calendars, contacts, SharePoint lists etc. This will require Multiple Site Hierarchy.
    • If business requirements drive design for personalized web parts, data views, dashboards driven by user identity, it may require Single Site or Few Sites based on site types.
• Navigation – Cross Site  Navigation and Cross Site-Collection Navigation
• Site Life Cycle Management
  • Needs to define process for  New Site Provisioning and Site Decommissioning
    • How does site would be provisioned? IT managed; User Managed through IT defined workflow, User Managed through browser based site templates etc.
    • Define business process to request provisioning new sites – both in bulk & individual
      • Site Decommissioning Process – Consider archiving site, instead of deleting it
      • Consider Auto Site Provisioning and Decommissioning Process
  • Needs to define process of extending or maintaining existing sites with new features
  • Site Auditing – Build tools to audit site provisioning, site membership, site maintenance, and  site decommissioning
  • Multiple ways to define site templates in SharePoint – Site Definitions,  Feature Stapling, Web Templates, Coded Site Templates based on blank site templates and activating/maintaining features programmatically
    • One way to speed up initial site design – Use out of box site templates (e.g. team site or blank site) with browser based customizations to speed up initial site template design working with business owners, Save site as template, and import saved site template wsps into Visual Studio to create base Site Template. This process would work only for non-publishing sites. Publishing feature disables save as site template.

Content – Site and Page Contents Considerations

• Page Design – Page Templates – Content Pages
  • Site Pages vs. Application Pages vs. Page Layouts
    • Site Pages – If users are expected to add/remove web parts, personalize page, or requires web parts
    • Application Pages – Administrative Pages
    • Page Layouts – If users are expected to manage contents on page or users are expected to create pages based on pre-defined formats.
  • For the publishing driven sites, needs to define content approval process, content authoring process, and content deployment strategies
• Collaborative Content
  • Collaboration with Customers – Document Libraries, Annoucements, Calendar, Contacts, Team Sites
  • Rich Media – Audios and Videos, should define Digital Asset Management strategies
• Rollup Views
  • Content Query Web Part – within site collection
  • Lightning Conductor Third-party web parts – cross site collection
  • Custom Search Based API - cross site collection
• Data – Integration with other systems within organization
  • Define systems to integrate – SAP, CRM, Lotus Notes, Other SharePoint Farms (e.g. IT Intranet, Document Warehouses), and Third Party Systems
  • Each System would provide its own challenge to access data from SharePoint, May require developing custom web services interface or BCS for platform Integration
  • Does external users requires data interactivity – Reporting, KPIs, Scorecards, Dashboards etc.? Do external user’s credentials pass through to the Business intelligence systems? – This may require SSRS, Excel Services, Performance Point Services, Visio Services, BCS or other mechanisms with Kerberos or Claims enabled authentication
  • Data Security – Define process to expose internal data securely to the customers
    • Would customer credentials  pass through to the internal systems? – this would require Kerberos enabled on the SharePoint
    • Access Internal Systems based on User/Site Metadata/Personalization and Service Accounts instead of passing user credentials to the data source systems, requires proper metadata governance, metatada mapping, and metadata sync process
• Search
  • Decide to use Fast Search vs. Enterprise Search capability vs. Custom Search Driven Components
  • Searching data from multiple internal systems may require BCS/LOB connectivity for platform integration with metadata targeted custom search API
  • Allows you to target information to customer by external user expertise and based on user profiles

Other Major Considerations

• User Personalization and Preferences
  • Define User Personalization Data Store – SharePoint User Profiles vs. SQL Server Users DB vs. Custom Tools
  • Use User Metadata to target specific contents and implement personalization
  • May require tools to Sync User Metadata with Source Systems
  • May require tools to manage User Maintained Metadata and Preferences
• Metadata
  • Application Metadata – Store in web.config, web application configuration store etc.
  • Site Metadata – Store in SharePoint site property bag properties
  • User Metadata – Store in User Profiles, SQL Servers, and AD/ADLDS properties etc.
• Licensing
  • Work with your Microsoft reps for licencing impact, Each organization would affect different way
  • Per User CAL – Internal vs. External Facing
• Social Integration
  • Any Social Integration – Twitter, Facebook, LinkedIn, Google+ etc.
• Mobility Access
  • Target Platform – Blackberry, IPhone, Android, Windows Mobile
  • Any support for Mobile Device Access, HTML 5, MAC OS, iOS for cross-platforms and cross-device support.
  • Plant to integrate open standards like Jquery, Avoid Plugins like Adobe Flash or Silverlight for UI which not supported on iOS as of now
• Cross-Browser Support
  • Define Browser Support Standards for IE, Chrome, Firefox – Checkout SharePoint 2010 Level 1 and Level 2 browser support and see if any custom tools needs to incorporated
  • Do you really need to use Silverlight or Adobe Flash? May be HTML 5, CSS 3 for industry standards
  • Target Standard Screen Resolution – 1024×768 vs. 1280×1024
• Look and Feel – Branding
  • Custom Master Pages, CSS, Images, JavaScript, jQuery files etc.
  • UX Experience – AJAX vs. Jquery vs. Silverlight vs. HTML5 vs. JavaScript
  • Concept Design to Wireframes – Design Wireframes for pages, sub sites, and content pages
  • Style Guide – Microsoft Metro look & feel vs. Corporate Style Guide
• Custom Development - Methodology and Environments
  • Build out Multiple Environments – Individual Developer VMs, Integration, Staging, Authoring, Production
  • Implement Coding Guidelines and Adhere Standards
  • Plan to standardize Code Organization in Visual Studio – Many Codeplex tools available to enable RAD
  • Plan to Use Source Code Control Management like TFS
  • Plan to perform Unit Testing, Automated Build Management, and Continuous Integration for Proper Release Management.
  • Plan to standardize Code Deployment using PowerShell Scripts vs. Manual PS Commands – Packaging using Features & Solutions Framework
• Production Diagnosis – Logging and Auditing
  • Review out of box diagnostics and logging options – ULS, Event Logs, Developer Dashboards
  • Plan to build IT Support and Monitoring Framework – Error Handling, Logging
• Performance
  • Caching – ASP.NET Caching vs. Page Output Caching vs. Custom Caching Components
  • Browser Optimizations – CSS Optimizations
  • Plan to perform Load Testing
• Anti-Virus
  • Plan to use SharePoint specific Anti-Virus product to scan external user uploaded documents.
  • Consider blocking Infected Documents
• Localization – Global Platform
  • Decide to use Different UI experience for different regions or  Consistent UI experience at all regions
  • Multi- Lingual Sites vs. MUI vs. Both vs. ASP.NET Custom Globalization Resource Files
    • Variations and Content Translation Tools
    • Sites in specific language and currency
• Web Analytics
  •  SharePoint Out of box Web Analytics or Custom ISV tools
  • SharePoint Web Analytics not useful – Per Site Collection or Per Site, Instead Integrate with Web Trend or Google Analytics or ISV tools
• End-User Training and Adoption
  • Plan to have proper documentation, online help, and system adoption plans
  • Plan to have proper communication and notifications for updates or new features rollout
  • Plan to have initial Pilot program, product roadshow, or adoption programs
• IT Support and Monitoring
  • Plan to have feedback forums for external users to submit incidents and general system help
  • Plan to have dedicated IT support team to respond user incidents in timely manner

Decision Time – Deactivate SharePoint Foundation Web Application Service on Central Admin or Deploy Custom Solutions from Central Admin

Note: This article only applies to central admin server used for Application Tier in SharePoint Farm.

I recently came across very interesting error while deploying solutions and activating features using powershell on one of our farm’s central admin server. What surprised me that we were using same approach to deploy our code from central admin server since last couple of months and suddenly it’s stopped working while activating features and throwing error.

Enable-SPFeature : The Feature is not a Farm Level Feature and is not found in a Site level defined by the Url. At D:\Deploy\SPSolutionDeploymentScript.ps1:287 char:22 + Enable-SPFeature <<<<  –identity $webFeatureName -URL $spWeb.url -Confirm:$false + CategoryInfo: InvalidData: (Microsoft.Share…etEnableFeature:SPCmdletEnableFeature) [Enable-SPFeature], SPCmdletException + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletEnableFeature

Looking at the error, it was clear that PowerShell wasn’t able to find SharePoint Features and Solutions framework API on the server. What I didn’t know was what would or which SharePoint Service application would enable this framework on the server. My first guess was to reach out to our SharePoint admin to see if he is aware of any recent changes on the central admin server configuration. Additionally, I tried to deploy and activate features through one of the WFE servers and it worked fine. As I was waiting for admin response, I have reached out to greater SharePoint community via twitter. My good friends from SharePoint Twitter communities, both Dan Usher (@usher) and Clayton Cobb (@warrtalon) came to rescue right away and first clue was SharePoint Foundation Web Application Service may not be running. While I was trying to confirm whether this service was running earlier and stopped recently causing feature activation issues, I received response from Admin that this service was indeed stopped and he may have stopped recently.

Well, problem is solved and resolution is clear. We just needed to activate the SharePoint Foundation Web Application Service to resolve the issue. But, as we were exchanging information on twitter, I have realized that this could be major SharePoint release management decision. As an Admin, we would like to disable the SharePoint Foundation Web Application Service on the central admin server. Disabling SharePoint Foundation Web Application Service on central admin server seems one of the best practices since it isn’t used to serve pages to the end-users and disabling this service would conserve server memory for other dedicated SharePoint application tier services enabled on Central Admin/Application Server.

In General, here are the guidelines I have came to conclusion whenever I come across similar situation in future.

• It is still best approach to deploy code and activate features from central admin. This would allow central admin server as a main administrative consoles for both configurations and custom deployment.
• It is still best practice to disable the SharePoint Foundation Web Application Service on the Central Admin Server to avoid additional performance overhead by running less SharePoint Web Application IIS worker processes
• Since you have to activate SharePoint Foundation Web Application Service on the Central Admin Server to deploy code from the Central Admin Server, It would be great practice to enable the Service during deployment process and disable during normal runtime. It would be nice to have a deployment tasks to enable the service, deploy custom solutions, and disable the service.
• One last point, there is nothing written on stone or as a best practice to deploy code from central admin, it is just my preferred method to centralize administrative tasks in one place. If your situation is different and able to deploy custom solutions from the WFE servers running SharePoint Foundation Web Application Service, you are covered.

Here is another great article recommendaed by Dan Usher and it provides same architectural insights faced by SharePoint Architects and IT Pros in real world – http://blogs.technet.com/b/speschka/archive/2010/11/27/beware-of-default-solution-deployments-for-custom-claims-providers-in-sharepoint-2010.aspx

Best Practices to Change App Pool Account for SharePoint Web Application

Updating SharePoint Web Application Pool is one of the most common actions for SharePoint administration. I have repeatedly seen many SharePoint administrators and my fellow colleagues updating their SharePoint web application pool in the IIS and later realizing that their SharePoint content application is inaccessible and throws “Cannot connect to the configuration database” error.

The real reason behind this is when you create web application either through PowerShell or central admin, SharePoint configures application pool information at many different locations including machine level permissions, IIS, and database permissions. If you ever want to manually change the application pool, you must be aware of what really happens under the hood and visit all the different locations to change application pool manually. As you may think, manually changing all these machine level settings is tedious, error-prone, and requires better option. Luckily Microsoft has provided better option as manage service accounts page on the central administration site. It is best practice to change content web application pool or even service web application pool from the central administration to ensure SharePoint Content Web application runs smoothly.

You can use following step by step guide to change application pool for the given SharePoint web application. Additionally, it would walk you through what really happens under the hood and where SharePoint makes necessary changes to ensure Application Pool is configured properly.

Pre-requisites

• New AppPool account must be Domain User Account (e.g. Niks\SPAppPool)
• New AppPool account must register as SharePoint Managed Account

Changing Application Pool from the Central Administration

Visit Manage Service Accounts page on the central administration to change the application pool.

Run the IISReset after updating application pool to ensure all the configuration settings has been updated to access SharePoint Web Application correctly.

What really happens under the hood?

After you change the application pool through central administration, SharePoint automates various configuration settings changes at the machine level, IIS, and SQL Server.

• SharePoint Web Application App Pool in IIS

  

• Machine-level Permissions
  • New AppPool account added as Member in the WSS_WPG, AD Group
  • New AppPool account added as Member in the built-in IIS_IUSRS, AD Group

               

• SQL Server and database permissions
  • SharePoint will create new SQL Server Login for AppPool Account in the Database if it doesn’t exists
  • New AppPool account is assigned to the db_owner role for the Web application content databases.

  

  • New AppPool account is assigned to the WSS_CONTENT_APPLICATION_POOLS role associated with the farm configuration database.

  

  • New AppPool account is assigned to the WSS_CONTENT_APPLICATION_POOLS role associated with the SharePoint_Admin content database.

  

  • New AppPool account is assigned to the WSS_CONTENT_APPLICATION_POOLS role associated with the StateService database.

  

  • New AppPool account will be assigned to the db_owner role for the associated user profile service application databases (e.g. Profile DB, Social DB, and Sync DB)

There you go. Regardless of what you do and where you manually change application pool account info, you still have to change application pool through manage security accounts screen. So, why not just change only at 1 place on manage security accounts screen and let SharePoint does it’s magic to update all the required places. Hopefully this will help. !!!!!

Step by Step Guide to Configure SharePoint 2010 Forms Based Authentication with SQL

It is very common to use SQL Server database to store external users and roles in extranet environments for physical separation of the internal and external users. Typically external identity systems require specific schema changes and AD administrators don’t allow applications to store their users in the main organization domain directory for security concerns.

I have recently wrote an article on step by step guide to configure SharePoint 2010 FBA with ADLDS. This article follows same pattern to configure SharePoint FBA with SQL Server. As you may notice, most of the steps are same and web configuration file changes are similar. This article describes 5-Steps guide to configure SQL Users and Roles in Single-Server SharePoint 2010 environment on Windows 2008 R2 server for Forms Based Authentication.

Note: If you are looking for detailed step by step guide with lots of screenshots, you can download 35-pages step by step PDF guide demonstrating same steps discussed in this article.

 Step 1 – Create SQL Server Database to host FBA Accounts

• Run C:\Windows\Microsoft.NET\Framework\v2.0.50727\aspnet_regsql.exe to create the SQL DB – aspnetdb
• Verify that Niks\Administrator is running Application Pools for the Content and Central Admin Web Application and it’s added as dbowner role on the aspnetdb database.

Step 2 – Add SQL Users and Roles for FBA

• Download the MembershipSeeder to load users for form-based authentication – http://cks.codeplex.com/releases/view/7450
• How to use MembershipSeeder tool?
• Open the  \Bin\Release\MembershipSeeder.exe and Update the SQL Server Name by clicking “Configure” button and close the tool
• Reopen the tool and use following guidelines to add users.
• Add 1 user at a time from membership section using “create” button
• Add 1 role at a time from roles section using “create” button
• Add 1 user to the role at a time from roles section using “Add to Role” button

• Add SQL Roles and Users as following
• Roles – sqlowners, sqlcontributors, sqlreaders
• Users – sqlowner, sqlcontributor, sqlreader and add them to specific groups

Step 3 – Create New Web Application with Forms Based Authentication

• Add DNS entries for the host headers – sqlportal.niks.local
• Create New Web Application with Claims Based Authentication
• Specify Port-80 and Host Header – sqlportal.niks.local
• Select Windows Authentication and Forms Based Authentication
• Specify Membership Provider – SqlMember and Role Provider – SqlRole
• Specify proper content database name and leave everything else as it is
• Create New Site Collection and specify Niks\Administrator as Site Collection Admin
• Verify the Windows Authentication by logging to http:\\sqlportal.niks.local as Using Niks\Administrator

Step 4 – Update the Web Config Files for FBA – Content Web App, Central Admin Web App, and STS

• Update the Web Config Files for FBA, Please make sure to have a copy of all the original web.config files before making changes.
• Follow Mirjam’s blog, it works – http://www.sharepointchick.com/archive/2010/05/07/configuring-claims-and-forms-based-authentication-for-use-with-a.aspx
• Content Web App Configuration
• Update sqlportal.niks.local web application web.config – C:\inetpub\wwwroot\wss\VirtualDirectories\sqlportal.niks.local80\web.config
• Find the </configSections> entry and add following XML directly below i

  <connectionStrings>
     <clear />
     <add name="AspNetSqlMembershipProvider" connectionString="data source=SP2010VM;Integrated Security=SSPI;Initial Catalog=aspnetdb"  providerName="System.Data.SqlClient" />
  </connectionStrings>

• Replace the  <PeoplePickerWildcards> entry with following XML

  <PeoplePickerWildcards>
   <clear />
   <add key="AspNetSqlMembershipProvider" value="%" />
   <add key="SqlMember" value="%"/>
   <add key="SqlRole" value="%"/>
  </PeoplePickerWildcards>

• Locate the <membership> entry and Replace everything from <membership> to </membership> with the following XML

   <membership defaultProvider="i">
      <providers>
        <clear />
        <add connectionStringName="AspNetSqlMemberShipProvider"
           enablePasswordRetrieval="false"
           enablePasswordReset="true"
           requiresQuestionAndAnswer="true"
           passwordAttemptWindow="10"
           applicationName="/"
           requiresUniqueEmail="false"
           passwordFormat="Hashed"
           name="SqlMember"
           type="System.Web.Security.SqlMembershipProvider, System.Web, Version=2.0.0.0,
  Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
        <add name="i" type="Microsoft.SharePoint.Administration.Claims.SPClaimsAuthMembershipProvider, Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
     </providers>
    </membership>

• Locate the <roleManager> entry and Replace everything from <roleManager> to </roleManager> with the following XML

   <roleManager defaultProvider="c" enabled="true" cacheRolesInCookie="false">
      <providers>
          <clear />
           <add connectionStringName="AspNetSqlMemberShipProvider"
              applicationName="/"
              name="SqlRole"
              type="System.Web.Security.SqlRoleProvider, System.Web, Version=2.0.0.0,
  Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
           <add name="c" type="Microsoft.SharePoint.Administration.Claims.SPClaimsAuthRoleProvider,
  Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
     </providers>
  </roleManager>

• Central Admin Web Application 
  • Update Central Admin Web Application web.config file (to find central admin web.config – go to the IIS, select central admin, and click Explore to find contents)
  • Find the </configSections> entry and add following XML directly below it

  <connectionStrings>
       <clear />
       <add name="AspNetSqlMembershipProvider" connectionString="data source=SP2010VM;Integrated Security=SSPI;Initial Catalog=aspnetdb"  providerName="System.Data.SqlClient" />
    </connectionStrings>

• Replace the  <PeoplePickerWildcards> entry with following XML

<PeoplePickerWildcards>
   <clear />
   <add key="AspNetSqlMembershipProvider" value="%" />
   <add key="SqlMember" value="%"/>
   <add key="SqlRole" value="%"/>
</PeoplePickerWildcards>

• Find the <system.web> entry and add the following XML directly below it. By default, there should be 1 blank Membership or RoleManager entry. Double check whether the <membership> and <rolemanager> entries only exist ones. Delete any double entries.

  <membership defaultProvider="i">     
    <providers>
        <clear />
        <add connectionStringName="AspNetSqlMembershipProvider"
           enablePasswordRetrieval="false"
           enablePasswordReset="true"
           requiresQuestionAndAnswer="true"
           passwordAttemptWindow="10"
           applicationName="/"
           requiresUniqueEmail="false"
           passwordFormat="Hashed"
           name="SqlMember"
           type="System.Web.Security.SqlMembershipProvider, System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
        <add name="i" type="Microsoft.SharePoint.Administration.Claims.SPClaimsAuthMembershipProvider, Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
   </providers>
 </membership>
      
 <roleManager defaultProvider="AspNetWindowsTokenRoleProvider" enabled="true" cacheRolesInCookie="false">
    <providers>
        <clear />
        <add connectionStringName="AspNetSqlMembershipProvider"
           applicationName="/"
           name="SqlRole"
           type="System.Web.Security.SqlRoleProvider, System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
        <add applicationName="/"
           name="AspNetWindowsTokenRoleProvider"
           type="System.Web.Security.WindowsTokenRoleProvider, System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    </providers>

 </roleManager>

• Modify STS web.config file 
  • From the IIS, select the SecurityTokenServiceApplication under SharePoint Web Services and click Explore – it should take you to the C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\WebServices\SecurityToken
  • Find the </system.net> entry and add the following XML directly below it

  <connectionStrings>
     <clear />     
     <add name="AspNetSqlMembershipProvider" connectionString="data source=SP2010VM;Integrated Security=SSPI;Initial Catalog=aspnetdb"  providerName="System.Data.SqlClient" />
  </connectionStrings>   
  <system.web>
    <membership defaultProvider="i">
     <providers>
        <clear />
       <add connectionStringName="AspNetSqlMembershipProvider"
          enablePasswordRetrieval="false"
          enablePasswordReset="true"
          requiresQuestionAndAnswer="true"
          passwordAttemptWindow="10"
          applicationName="/"
          requiresUniqueEmail="false"
          passwordFormat="Hashed"
          name="SqlMember"
          type="System.Web.Security.SqlMembershipProvider, System.Web, Version=2.0.0.0,
  Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
      <add name="i" type="Microsoft.SharePoint.Administration.Claims.SPClaimsAuthMembershipProvider, Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
     </providers>
   </membership>
  
   <roleManager defaultProvider="c" enabled="true">
     <providers>
     <clear />
        <add connectionStringName="AspNetSqlMembershipProvider"         applicationName="/"
           name="SqlRole"
           type="System.Web.Security.SqlRoleProvider, System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
        <add name="c" type="Microsoft.SharePoint.Administration.Claims.SPClaimsAuthRoleProvider, Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
     </providers>
   </roleManager>
  </system.web>   

Step 5 – Configure the SharePoint Authorization and Verify the Access for the both AD (Windows) and SQL(Forms Based) Users

• Verify AD Groups and Users are available to test
  • Sample Global Security Groups – adowners,adcontributors,adreaders
  • Sample Users – adowner,adcontributor,adreader
  • Add AD Groups and SQL Roles as SharePoint Security Groups – Readers, Contributors, and Owners
• Verify that central admin and content web application people picker finds both AD groups/users and SQL roles/users – Please note that SQL Roles can be searched with the full role name while SQL user names can be searched via wildcard

• Add AD users and SQL users into SharePoint Security Groups via AD Groups/SQL Roles and verify proper access – read-only, contribute, full control

Following screenshot demonstrate that I was able to successfully login as SQL User – SQLOwner

More Resources

• Forms Based Authentication in SharePoint 2010 using Sql Membership Providers – http://sharepoint-2010-world.blogspot.com/2011/03/forms-based-authentication-in.html
• Andy’s Blog
  • UAG – How to Authenticate users against SQL Server – http://microsoft-iag.blogspot.com/2010/10/uag-how-to-authenticate-users-against.html
  • UAG–Enable SSO by passing user credentials to formula based Web Applications – http://microsoft-iag.blogspot.com/2010/10/uagenable-sso-by-passing-user.html
• Great Article – It Woks – Mirjam van Olst
  • Configuring claims and forms based authentication for use with a SQL provider in SharePoint 2010 – http://www.sharepointchick.com/archive/2010/05/07/configuring-claims-and-forms-based-authentication-for-use-with-a.aspx
• Jeremy Jameson – CBA and SQL Server
  • http://blogs.msdn.com/b/jjameson/archive/2011/02/19/configuring-claims-based-authentication-in-sharepoint-server-2010.aspx
  • http://blogs.msdn.com/b/jjameson/archive/2011/02/25/claims-login-web-part-for-sharepoint-server-2010.aspx

Step by Step Guide to Configure SharePoint 2010 Forms Based Authentication with ADLDS

It is very common to use Active Directory Lightweight Directory Services – ADLDS in Windows 2008 environments (ADAM in Windows 2003 environments) to store external users in extranet environments for physical separation of the internal and external users. Typically external identity systems require specific schema changes and AD administrators don’t allow applications to store their users in the main organization domain directory for security concerns.

Recently I was able to successfully configure SharePoint 2010 Claims based authentication for the ADLDS using Forms based authentication. This article describes 5-Steps guide to configure ADLDS in Single-Server SharePoint 2010 environment on Windows 2008 R2 server for Forms Based Authentication in non-SSL environment. You can easily adopt this guide to use in SSL environment.

Note: If you are looking for detailed step by step guide with lots of screenshots, you can download 55-pages step by step PDF guide demonstrating same steps discussed in this article.

Step 1 – Configure ADLDS Environment

• Install ADLDS Server Role (http://technet.microsoft.com/en-us/library/cc754486(WS.10).aspx)
• Create ADLDS Instance (e.g. SP2010VM.niks.local:52983) and Partition (e.g. O=NIKSADLDS,C=LOCAL) (http://technet.microsoft.com/en-us/library/cc725619(WS.10).aspx)
• Connect to ADLDS Instance from ADSI EDIT Tool (http://technet.microsoft.com/en-us/library/cc732675(WS.10).aspx)
• Create Container – CN for Groups and Users (http://technet.microsoft.com/en-us/library/cc730701(WS.10).aspx)
• Create CN (e.g. WPSCN)
• Create Users (e.g. WPSCNAdminUser, WPSCNMemberUser, WPSCNReaderUser)
• Reset Passwords of all Users
• Enable Users
• Set ms-DS-UserAccountDisabled = FALSE
• Set ms-DS-UserDontExpirePassword = FALSE
• Set ms-DS-UserPasswordNotRequired = FALSE
• Make Sure User’s cn, distinguishedName, and userPrincipalName is filled out
• Make Sure userPrincipalName  has no spaces.
• Create Groups (e.g. WPSCN Admin Users, WPSCN Member Users, WPSCN Reader Users)
• Make Sure Group’s cn, distinguishedName is filled out
• Add Users to the Groups – Use Group’s Member Property

 Step 2 – Create New Web Application with Forms Based Authentication

• Add DNS entries for the host headers (e.g. adldsportal.niks.local)
• Create New Web Application with Claims Based Authentication
• Specify Port-80 and Host Header (e.g. adldsportal.niks.local)
• Select Windows Authentication and Forms Based Authentication
• Specify Membership Provider – LdapMember and Role Provider – LdapRole
• Specify proper content database name and leave everything else as it is
• Create New Site Collection and specify Niks\Administrator as Site Collection Admin
• Verify the Windows Authentication by logging to http:\\adldsportal.niks.local as Using Niks\Administrator

Step 3 – Grant “Application Pool” accounts READ Permission on ADLDS instance

• Add Content SharePoint Web Application “Application Pool”, Central Administration Web Application “Application Pool”, and Security Token Service “Application Pool”  accounts READERS Roles on ADLDS instance. This would allow SharePoint to browse ADLDS store in least privileged scenario.
• In My Sandbox, both ADLDS Administrators and SharePoint Application Pool accounts are same so, there is no need for explicit setting of permissions but in real world least privileged environment, these application pool accounts will be different and must be granted permission on the ADLDS. Failure of granting permissions for STS application Pool account may cause login failure issues. Failure of granting permissions for Web Application Pool accounts may cause people picker failure.

Step 4 – Update the Web Config Files for FBA – Content Web App, Central Admin Web App, and STS

• Follow Mirjam’s blog, it works – http://sharepointchick.com/archive/2010/05/06/configuring-claims-and-forms-based-authentication-for-use-with-an.aspx
• Before you make any changes, Please make sure to have a copy of all the original web.config files before making changes.
• Please note that these web.config entries would work for ADLDS Containers, ADLDS Group Objects, and ADLDS Users Objects. If you are using either OUs, Persons, or other ADLDS objects, please modify the LDAP membership and roles providers entries as needed.
• Content Web App Configuration
• Update adldsportal.niks.local web application web.config – C:\inetpub\wwwroot\wss\VirtualDirectories\adldsportal.niks.local80\web.config
• Replace the  <PeoplePickerWildcards> entry with following XML

      <PeoplePickerWildcards>
        <clear />
        <add key="AspNetSqlMembershipProvider" value="%" />
        <add key="LdapMember" value="*" />
        <add key="LdapRole" value="*" />
      </PeoplePickerWildcards>

• Locate the <membership> entry and Replace everything from <membership> to </membership> with the following XML

      <membership defaultProvider="i">
        <providers>
          <clear />
          <add name="LdapMember" type="Microsoft.Office.Server.Security.LdapMembershipProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" server="SP2010VM.niks.local" port="52983" useSSL="false" userDNAttribute="distinguishedName" userNameAttribute="userPrincipalName" userContainer="CN=WPSCN,O=NIKSADLDS,C=LOCAL" userObjectClass="user" userFilter="(ObjectClass=user)" scope="Subtree" otherRequiredUserAttributes="cn" />
          <add name="i" type="Microsoft.SharePoint.Administration.Claims.SPClaimsAuthMembershipProvider, Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
        </providers>
      </membership>

• Locate the <roleManager> entry and Replace everything from <roleManager> to </roleManager> with the following XML

      <roleManager defaultProvider="c" enabled="true" cacheRolesInCookie="false">
        <providers>
          <clear />
          <add name="LdapRole" type="Microsoft.Office.Server.Security.LdapRoleProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" server="SP2010VM.niks.local" port="52983" useSSL="false" groupContainer="CN=WPSCN,O=NIKSADLDS,C=LOCAL" groupNameAttribute="cn" groupMemberAttribute="member" dnAttribute="distinguishedName" userNameAttribute="userPrincipalName" groupFilter="(ObjectClass=group)" userFilter="(ObjectClass=user)" scope="Subtree" />
          <add name="c" type="Microsoft.SharePoint.Administration.Claims.SPClaimsAuthRoleProvider, Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
        </providers>
      </roleManager>

• Central Admin Web Application
• Update Central Admin Web Application web.config file (to find central admin web.config – go to the IIS, select central admin, and click Explore to find contents)
• Replace the  <PeoplePickerWildcards> entry with following XML

      <PeoplePickerWildcards>
        <clear />
        <add key="AspNetSqlMembershipProvider" value="%" />
        <add key="LdapMember" value="*" />
        <add key="LdapRole" value="*" />
      </PeoplePickerWildcards>

• Find the <system.web> entry and add the following XML directly below it. By default, there should be 1 blank Membership or RoleManager entry. Double check whether the <membership> and <rolemanager> entries only exist ones. Delete any double entries.

    <membership defaultProvider="i">
      <providers>
        <clear />
        <add name="LdapMember"
   type="Microsoft.Office.Server.Security.LdapMembershipProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
   server="SP2010VM.niks.local"
   port="52983"
   useSSL="false"
   userDNAttribute="distinguishedName"
   userNameAttribute="userPrincipalName"
   userContainer="CN=WPSCN,O=NIKSADLDS,C=LOCAL"
   userObjectClass="user"
   userFilter="(ObjectClass=user)"
   scope="Subtree"
   otherRequiredUserAttributes="cn" />
        <add name="i" type="Microsoft.SharePoint.Administration.Claims.SPClaimsAuthMembershipProvider, Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
     </providers>
    </membership>
  
  <roleManager defaultProvider="AspNetWindowsTokenRoleProvider" enabled="true" cacheRolesInCookie="false">
        <providers>
          <clear />
         <add name="LdapRole"
   type="Microsoft.Office.Server.Security.LdapRoleProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
   server="SP2010VM.niks.local"
   port="52983"
   useSSL="false"
   groupContainer="CN=WPSCN,O=NIKSADLDS,C=LOCAL"
   groupNameAttribute="cn"
   groupMemberAttribute="member"
   dnAttribute="distinguishedName"
   userNameAttribute="userPrincipalName"
   groupFilter="(ObjectClass=group)"
   userFilter="(ObjectClass=user)"
   scope="Subtree" />
        <add applicationName="/"
           name="AspNetWindowsTokenRoleProvider"
           type="System.Web.Security.WindowsTokenRoleProvider, System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
       </providers>
  </roleManager>

• Modify STS web.config file
• From the IIS, select the SecurityTokenServiceApplication under SharePoint Web Services and click Explore – it should take you to the C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\WebServices\SecurityToken
• Find the </system.net> entry and add the following XML directly below it

<system.web> 
   <membership defaultProvider="i">
      <providers>
         <clear />
         <add name="LdapMember"
    type="Microsoft.Office.Server.Security.LdapMembershipProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
    server="SP2010VM.niks.local"
    port="52983"
    useSSL="false"
    userDNAttribute="distinguishedName"
    userNameAttribute="userPrincipalName"
    userContainer="CN=WPSCN,O=NIKSADLDS,C=LOCAL"
    userObjectClass="user"
    userFilter="(ObjectClass=user)"
    scope="Subtree"
    otherRequiredUserAttributes="cn" />
       <add name="i" type="Microsoft.SharePoint.Administration.Claims.SPClaimsAuthMembershipProvider, Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
      </providers>
   </membership>
   
   <roleManager defaultProvider="c" enabled="true">
      <providers>
    <clear />
    <add name="LdapRole"
     type="Microsoft.Office.Server.Security.LdapRoleProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
     server="SP2010VM.niks.local"
     port="52983"
     useSSL="false"
     groupContainer="CN=WPSCN,O=NIKSADLDS,C=LOCAL"
     groupNameAttribute="cn"
     groupMemberAttribute="member"
     dnAttribute="distinguishedName"
     userNameAttribute="userPrincipalName"
     groupFilter="(ObjectClass=group)"
     userFilter="(ObjectClass=user)"
     scope="Subtree" />
         <add name="c" type="Microsoft.SharePoint.Administration.Claims.SPClaimsAuthRoleProvider, Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
      </providers>
   </roleManager>
</system.web>

Step 5 – Configure the SharePoint Authorization and Verify the Access for the both AD (Windows) and ADLDS (Forms Based) Users

• Verify ADLDS Users and Groups are available to test
• Sample Roles – adldsowners, adldscontributors,adldsreaders
• Sample Users – adldsowner,adldscontributor,adldsreader and add them to specific groups

• Verify AD Groups and Users are available to test
• Sample Global Security Groups – adowners,adcontributors,adreaders
• Sample Users – adowner,adcontributor,adreader

• Verify that central admin people picker finds for both AD groups/users and ADLDS roles/users – Please note that ADLDS roles can be searched with the full role name while ADLDS user names can be searched via wildcard
• Configure AD Groups/Users and ADLDS Roles/Users Membership to the SharePoint Security Groups – Readers, Contributors, and Owners
• Verify that AD users and ADLDS users added in SharePoint Security Groups via AD Groups/ADLDS Roles have proper access – read-only, contribute, full control

Following screenshot demonstrate that I was able to successfully login as ADLDS User – WPSCNAdminUser

More Resources

• SharePoint 2010 and ASP.NET Form Authentication with ADAM
• ADAM-Forms Based Authentication in SharePoint 2010 – http://sharepoint-2010-world.blogspot.com/2011/03/adam-forms-based-authentication-in.html

• SharePoint 2010 and ASP.NET Form Based Authentication with ADLDS
• Installing the Active Directory Lightweight Directory Services [AD-LDS] – http://sharepoint-2010-world.blogspot.com/2011/03/installing-active-directory-lightweight.html
• AD LDS Getting Started Step-by-Step Guide – http://technet.microsoft.com/en-us/library/cc770639(WS.10).aspx
• ADLDS Forms Based Authentication in SharePoint 2010
• http://sharepoint-2010-world.blogspot.com/2011/03/adlds-forms-based-authentication-in.html
• ADLDS Forms Based Authentication in SharePoint 2007
• http://sharepointgeorge.com/2009/ad-lds-sharepoint-and-forms-based-authentication/
• http://www.gk.id.au/2009/05/ad-lds-sharepoint-and-forms-based.html
• http://leedale.wordpress.com/2008/09/11/moss-2007-forms-based-authentication-using-ad-lds-and-windows-server-2008/

Automated SharePoint 2010 Farm Level Solution Deployment and Retraction Process – Part IV – Using PowerShell Script

Article Series Content

• Basics of Farm Level SharePoint Deployment and Retraction
• Step by Step Building Deployment and Retraction PowerShell Script
• Final Version of Deployment and Retraction PowerShell Script
• Step by Step Using PowerShell Script in Batch Files (This Article)

This is fourth and final part of this article series and it provides step by step walk through on how to run deployment and retraction PowerShell script using batch files. Please note that even though this article walks through with batch files, you can still use PowerShell scripts directly to standardize the deployment process without batch files. Follow earliar articles in this series here at Part I, Part II, and Part III.

As I have described in earlier article, one of the assumptions in this script is all the batch files, log files, and WSPs exists in same directory during deployment process. If you are planning to have WSPs and log files in different directories other than batch files, you will have to update the script to make sure it’s not aware of current directory and manually pass in the full path for WSPs and Log files.

Let’s walk through final step by step walk through on how to create batch files and automate the custom SharePoint solution deployment and retraction process.

Step 1 – Prepare for Deployment and Retraction

• Create Deployment Directory – (e.g. C:\CustomSolutions\Deployment\CustomHomePage)
• Create Two Batch Files, Each for deployment and retraction (e.g. DeploySolution.bat and RetractSolution.bat)
• Copy this PowerShell script from Part III to deployment directory (e.g. SPSolutionDeploymentScript.ps1).
• Copy packaged SharePoint custom solution in same deployment directory (e.g. CustomHomePage.wsp)

Step 2 – Deployment Step

• Copy following commands in the Deployment batch file (e.g.DeploySolution.bat). You may notice that first step in the batch file is to bypass the PowerShell execution policy. By default, PowerShell execution policy is retracted which would restrict loading PS1 files from the batch files and this step would bypass the PowerShell execution policy. Next step is to call the SPSolutionDeploymentScript.ps1 and passed in all the required parameters.

powershell -Command “& {Set-ExecutionPolicy bypass}” -NoExit
powershell -Command “& {.\SPSolutionDeploymentScript.ps1 -solutionName:CustomHomePage.wsp -webAppUrl:http://sp2010vm:1000 -siteUrl:http://sp2010vm:1000 -siteFeatureNames:@(‘CustomHomePage_CustomHomeWidgets’) -webFeatureNames:@(‘CustomHomePage_CustomHomePage’) -logFileName:DeploySolution.log -action:SD}” -NoExit

pause

 

• To run the batch file, right click on the batch file and run as administrator.

• This will run PowerShell script and deploy the solution and activate features on both site collection level and sub sites levels.

Step 3 – Retraction Step

• Copy following commands for Retract batch file (e.g. RetractSolution.bat)

powershell -Command “& {Set-ExecutionPolicy bypass}” -NoExit
powershell -Command “& {.\SPSolutionDeploymentScript.ps1 -solutionName:CustomHomePage.wsp -webAppUrl:http://sp2010vm:1000 -siteUrl:http://sp2010vm:1000 -siteFeatureNames:@(‘CustomHomePage_CustomHomeWidgets’) -webFeatureNames:@(‘CustomHomePage_CustomHomePage’) -logFileName:RetractSolution.log -action:SR}” -NoExit

pause

• To run the batch file, right click on the batch file and run as administrator. It will run PowerShell script and retract the solution and deactivate features from both site collection level and sub sites levels.

Transcript Logging
As you would see, both deployment and retraction process would create log files as transcripts. Having log files would be really helpful in the environment where administrators deploys the code in production environment and needed to communicate with developers if deployment or retraction process fails.

Automated SharePoint 2010 Farm Level Solution Deployment and Retraction Process – Part III – Final Version of PowerShell Script

Article Series Content

• Basics of Farm Level SharePoint Deployment and Retraction
• Step by Step Building Deployment and Retraction PowerShell Script
• Final Version of Deployment and Retraction PowerShell Script (This Article)
• Step by Step Using PowerShell Script in Batch Files

This is third part of the article series and it provides the final/full version of PowerShell script built in earliar article to automate the SharePoint 2010 Farm Level Solution deployment/retraction process. If you haven’t read the first and Second parts of this series, I would recommend checking it out here at Part I and Part II.

Here is the final version of PowerShell script. Next article in this series will walk through step by step process of how to use this PowerShell script in real world using batch files to automate the process.

<# 
.DESCRIPTION 
    - Run this script to deploy/retract the farm level solution and activate/deactivate features

.GOAL
 - Build the most generic farm level soution deployment script
 
.ASSUMPTION
 - Batch files, log files, and WSPs exists in same directory during deployment process
 
.MAJOR FEATURES IMPLEMENTED
 - Parameters passed in from the batch file
 - Loading SharePoint PowerShell Plugin - Check if its already been loaded
 - Logging of Powershell script execution transcript, New file gets created for each run
 - Solution Deploy or Retract in single click
 - Waiting for  timer job execution while deploying and retracting solution
 - Nullable $siteFeatureNames and $webFeatureNames parameters to support deployment only option, no activation of features
 - Support Multiple Site and Web Level Features - Site feature and web features are passed in as array paramters
 - Support Activation of features at Multiple Webs - Activate/Deactivate Web Level features on all webs in site collection
 - Check for webapplication level resources to deploy or retract solutions at the web application level or all web application level
 - Prompt the script execution details with different color codes - green for major steps, yellow for detailed info, and red for errors

#>

param (
 [string]$solutionName = "$(Read-Host 'Enter the Solution WSP Name. [e.g. Sample.wsp]')",
 [string]$webAppUrl = "$(Read-Host 'Enter the Web Application URL. [e.g. <a href="http://sp2010vm]'/">http://sp2010vm]'</a>)",
    [string]$siteUrl = "$(Read-Host 'Enter the Site Collection URL. [e.g. <a href="http://sp2010vm/sites/site]'">http://sp2010vm/sites/site]'</a>)",
 [string[]]$siteFeatureNames = $null, #"$(Read-Host 'Enter the Site Level Features. [e.g. @('Feature1', 'Feature2')]')",
 [string[]]$webFeatureNames = $null, #"$(Read-Host 'Enter the Web Level Features. [e.g. @('Feature1', 'Feature2')]')",
    [string]$logFileName = "$(Read-Host 'Enter the Log File Name. [e.g. Sample.log]')",
    [string]$action = "$(Read-Host 'Enter [SD] to deploy solutions or [SR] to retract the solution')"
)

# Defination of main function
function main() {
 # find the current directory
 $currentDirectory = Get-Location
   
 # delete existing logfile
 $logFilePath = "$currentDirectory\$logFileName"
 if (Test-Path $logFilePath)
 {
  Remove-Item $logFilePath
 }
 
 # create new log file and start logging
 Start-Transcript $logFilePath

 # check to ensure Microsoft.SharePoint.PowerShell is loaded
 $snapin = Get-PSSnapin | Where-Object {$_.Name -eq 'Microsoft.SharePoint.Powershell'}
 if ($snapin -eq $null)
 {
  Write-Host "Loading SharePoint Powershell Snapin"
  Add-PSSnapin "Microsoft.SharePoint.Powershell"
 }
 
 # deploy the solution
 if ($action -eq "SD")
 {
  Write-Host "Step 1 - Add Solution Package: " $solutionName -foregroundcolor Green
  AddSolution
  
  Write-Host "Step 2 - Deploy Solution Package: " $solutionName -foregroundcolor Green
  InstallSolution
  
  Write-Host "Step 3 - Timer Job to deploy the Solution Package" -foregroundcolor Green
  Wait4TimerJob

  Write-Host "Step 4 - Activate Site Collection Level Features" -foregroundcolor Green
  ActivateSiteFeatures
  
  Write-Host "Step 5 - Activate Web Level Features" -foregroundcolor Green
  ActivateWebFeatures
 } 
 
 # retract the solution
 if ($action -eq "SR")
 {
  Write-Host "Step 1 - Deactivate Web Level Features" -foregroundcolor Green
  DeactivateWebFeatures
  
  Write-Host "Step 2 - Deactivate Site Collection Level Features" -foregroundcolor Green
  DeactivateSiteFeatures
  
  Write-Host "Step 3 - Uninstall Solution Package: " $solutionName -foregroundcolor Green
  UnInstallSolution
  
  Write-Host "Step 4 - Timer Job to Retract the Solution Package" -foregroundcolor Green
  Wait4TimerJob
  
  Write-Host "Step 5 - Remove Solution Package: " $solutionName -foregroundcolor Green
  RemoveSolution
 }
 
 # stop the logging
 Stop-Transcript
}

# Add the solution package
# Adds a SharePoint solution package to the farm Solution gallery, It doesn't deploy the uploaded solution yet.
function AddSolution()
{
 $solution = Get-SPSolution | where-object {$_.Name -eq $solutionName}
 if ($solution -eq $null)
 {
  Write-Host "Adding solution package" -foregroundcolor Yellow
  $solutionPath = "$currentDirectory\$solutionName"
  Add-SPSolution -LiteralPath $solutionPath -Confirm:$false
 }
}

# Deploy the solution package 
# Deploys an installed SharePoint solution on all the WFE servers in the farm.
# Deploying solution in the farm installs the feature automatically. It installs all the features
# on each server at the farm, web, site collection, and site level but It doesn't activate them.
# Since it provisions the file on each WFE, it is a timer job.
function InstallSolution()
{
 $solution = Get-SPSolution | where-object {$_.Name -eq $solutionName}
 $solutionId = $solution.Id
 if ($solution -ne $null)
 {
  $solutionDeployed = Get-SPSolution -Identity $solutionId | where-object {$_.Deployed -eq "False"}
  if ($solutionDeployed -eq $null)
  {
   if ( $solution.ContainsWebApplicationResource )
   {
    Write-Host "Deploying solution package to web application: " $webAppUrl -foregroundcolor Yellow
    Install-SPSolution -Identity $solutionName -WebApplication $webAppUrl -GACDeployment -Confirm:$false
   }
   else
   {
    Write-Host "Deploying solution package to all web applications" -foregroundcolor Yellow
    Install-SPSolution -Identity $solutionName -GACDeployment -Confirm:$false
   }
  }
 }
}

# Activate the Site level features
function ActivateSiteFeatures()
{ 
 if ($siteFeatureNames -ne $null)
 {
  $spSite = Get-SPSite $siteUrl  
  foreach($siteFeatureName in $siteFeatureNames)
  {
   Write-Host "Trying to Activate Site Collection Level Feature: " $siteFeatureName -foregroundcolor Yellow
   $siteFeature = Get-SPFeature -site $spSite.url | where-object {$_.displayname -eq $siteFeatureName}
   if ($siteFeature -eq $null)
   {
    Write-Host "Activating Site Level Features at " $spSite.url -foregroundcolor Yellow
    Enable-SPFeature –identity $siteFeatureName -URL $spSite.url -Confirm:$false
   }
  }  
  $spSite.Dispose()
 }
}

# Activate the Web level features
function ActivateWebFeatures()
{
 if ($webFeatureNames -ne $null)
 {
  $spSite = Get-SPSite $siteUrl
 
  #Cycle through all webs in the collection and activate all the features
  foreach($spWeb in $spSite.AllWebs)
  { 
   foreach($webFeatureName in $webFeatureNames)
   {  
    Write-Host "Trying to Activate Web Level Features: " $webFeatureName -foregroundcolor Yellow
    $webFeature = Get-SPFeature -web $spWeb.url | where-object {$_.displayname -eq $webFeatureName} 
    if ($webFeature -eq $null)
    {
     Write-Host "Activating " $webFeatureName " at " $spWeb.url -foregroundcolor Yellow
     Enable-SPFeature –identity $webFeatureName -URL $spWeb.url -Confirm:$false
    }
   }
  }
  
  $spWeb.Dispose()
  $spSite.Dispose()
 }
}

# Deactivate the Web level features
function DeactivateWebFeatures()
{
 if ($webFeatureNames -ne $null)
 {
  $spSite = Get-SPSite $siteUrl
 
  #Cycle through all webs in the collection and deactivate all the features
  foreach($spWeb in $spSite.AllWebs)
  { 
   foreach($webFeatureName in $webFeatureNames)
   {  
    Write-Host "Trying to Deactivate Web Level Features: " $webFeatureName -foregroundcolor Yellow
    $webFeature = Get-SPFeature -web $spWeb.url | where-object {$_.displayname -eq $webFeatureName} 
    if ($webFeature -ne $null)
    {
     Write-Host "Deactivating " $webFeatureName " at " $spWeb.url -foregroundcolor Yellow
     Disable-SPFeature –identity $webFeatureName -URL $spWeb.url -Confirm:$false
    }
   }
  }
  
  $spWeb.Dispose()
  $spSite.Dispose()
 }
}

# Deactivate the Site level features
function DeactivateSiteFeatures()
{
 if ($siteFeatureNames -ne $null)
 {
  $spSite = Get-SPSite $siteUrl
  foreach($siteFeatureName in $siteFeatureNames)
  {
   Write-Host "Trying to Deactivate Site Collection Level Feature: " $siteFeatureName -foregroundcolor Yellow
   $siteFeature = Get-SPFeature -site $spSite.url | where-object {$_.displayname -eq $siteFeatureName}
   if ($siteFeature -ne $null)
   {
    Write-Host "Deactivating Site Level Features at " $spSite.url -foregroundcolor Yellow
    Disable-SPFeature –identity $siteFeatureName -URL $spSite.url -Confirm:$false
   }  
  } 
  $spSite.Dispose()
 }
}

# Retract the solution package
# Retracts a deployed SharePoint solution from the farm entirely for all web application or given web application.
# This step removes files from all the front-end Web server.
# Please note that retracting solution in the farm uninstalls the feature automatically, if it hasn't uninstalled using UnInstall-SPFeature.
# Since it removes the file on each WFE, it is a timer job.
function UnInstallSolution()
{
 $solution = Get-SPSolution | where-object {$_.Name -eq $solutionName}
 $solutionId = $solution.Id
 if ($solution -ne $null)
 {
  $solutionDeployed = Get-SPSolution -Identity $solutionId | where-object {$_.Deployed -eq "True"}
  if ($solutionDeployed -ne $null)   
  { 
   if ( $solution.ContainsWebApplicationResource )
   {
    Write-Host "Retracting solution package from web application: " $webAppUrl -foregroundcolor Yellow
    UnInstall-SPSolution -Identity $solutionName -WebApplication $webAppUrl -Confirm:$false
   }
   else
   {
    Write-Host "Retracting solution package from all web applications" -foregroundcolor Yellow
    UnInstall-SPSolution -Identity $solutionName -Confirm:$false
   }
  }
 }
}

# Remove the solution package
# Deletes a SharePoint solution from a farm solution gallery
function RemoveSolution()
{
 $solution = Get-SPSolution | where-object {$_.Name -eq $solutionName}
 if ($solution -ne $null)
 {
  Write-Host "Deleting the solution package" -foregroundcolor Yellow
  Remove-SPSolution $solutionName -Confirm:$false
 }
}

# Wait for timer job during deploy and retract
function Wait4TimerJob()
{
 $solution = Get-SPSolution | where-object {$_.Name -eq $solutionName}
 if ($solution -ne $null)
 {
  $counter = 1  
  $maximum = 50  
  $sleeptime = 2 
  
  Write-Host "Waiting to finish soultion timer job"
  while( ($solution.JobExists -eq $true ) -and ( $counter -lt $maximum ) )
  {  
   Write-Host "Please wait..."
   sleep $sleeptime 
   $counter++  
  }
  
  Write-Host "Finished the solution timer job"   
 }
}

#Run the Main Function
main

 

Automated SharePoint 2010 Farm Level Solution Deployment and Retraction Process – Part II – Building PowerShell Script

Article Series Content

• Basics of Farm Level SharePoint Deployment and Retraction
• Step by Step Building Deployment and Retraction PowerShell Script (This Article)
• Final Version of Deployment and Retraction PowerShell Script
• Step by Step Using PowerShell Script in Batch Files

This is second part of the article series and it provides the step by step process of building PowerShell script to automate the SharePoint 2010 Farm Level Solution deployment/retraction process. If you haven’t read the first part of this series, I would recommend checking it out here.

Since PowerShell 2.0 is one of the pre-requisites, SharePoint Developer VM typically contains bits of PowerShell 2.0. Although Microsoft provides PowerShell 2.0 IDE for custom scripts, I like to use PowerGUI from Quest for PowerShell development - http://powergui.org/index.jspa.  Please check it out. It’s free download and provides nice debugging capabilities.

When I started creating PowerShell scripts to automate the SP2010 custom solution deployment/retraction process, one of my main goals was to build most generic farm level solution deployment script which would not only deploy/retract solutions and activate/deactivate features at the site collection level but also support activation/deactivation at all the sub sites level. If you are planning to deploy/retract solution at single site collection level, you can use this script right away without any changes. If you have more than one site collection, it would require slight modification.

Apart from most generic script, some of major features I wanted to support in this PowerShell script are as below.

• It must support Parameters. Most of the environment specific information must be parameterized.
• It should load SharePoint PowerShell Plugin automatically. If you have used SharePoint PowerShell Shell, it automatically loads this plugin but from custom windows PowerShell script, it must load manually and must check if it’s already loaded to avoid exceptions.
• Since one of the goals of this script is to handover the deployment to SharePoint admins, Logging of PowerShell script execution transcript is key. It would create new file for each run. If Admins ever informs developers that your script has been failed, you can ask for this log file for troubleshoot
• Major goal of this automated script to deploy or retract solutions in single click. Either you run PS manually or run using batch files; it should deploy/retract solutions and activate/deactivate features in single click.
• As many of you know, during SharePoint Solution deployment and retraction process, SharePoint creates timer job to deploy and retract solutions from all the WFEs. We need to ask PowerShell to wait until these timer jobs complete their tasks. This PowerShell script should support waiting for timer job execution while deploying and retracting solution.
• Support of Nullable $siteFeatureNames and $webFeatureNames parameters for deployment only option, no activation of features. Additionally you can pass one of these parameters, if Site or Web Level features are not applicable. Not all solutions would contain both web and site level features.
• Support Multiple Site and Web Level Features – Site feature and web features are passed in as array parameters
• Support Activation of features at Multiple Webs – Activate/Deactivate Web Level features on all webs in site collection
• Check for web application level resources to deploy or retract solutions at the web application level or all web application level
• Prompt the script execution details with different color codes – green for major steps, yellow for detailed info, and red for errors. If you are running these scripts manually on the server, you should be able to see the execution messages with different color codes.

To create the PowerShell script to automate Solution deployment/retraction process, please follow these steps. You can access final version of this script as 3rd part of this series here. Please note that I have posted this full working copy of this PowerShell script at TechNet Scripts library and you can find it here.

Step 1 – Define the Parameters to passed in PowerShell script

Define parameters to passed in web application URL, site collection URL, solution WSP Name, Site Level Features, Web Level Features, Log file name, and Action.  Last parameter would determine whether it’s deployment or retraction process. Ask calling application to enter SD for solution deployment and SR for solution retractions.

param (
 [string]$solutionName = "$(Read-Host 'Enter the Solution WSP Name. [e.g. Sample.wsp]')",
 [string]$webAppUrl = "$(Read-Host 'Enter the Web Application URL. [e.g. <a href="http://sp2010vm]'/">http://sp2010vm]'</a>)",
 [string]$siteUrl = "$(Read-Host 'Enter the Site Collection URL. [e.g. <a href="http://sp2010vm/sites/site]'">http://sp2010vm/sites/site]'</a>)",
 [string[]]$siteFeatureNames = $null, #"$(Read-Host 'Enter the Site Level Features. [e.g. @('Feature1', 'Feature2')]')",
 [string[]]$webFeatureNames = $null, #"$(Read-Host 'Enter the Web Level Features. [e.g. @('Feature1', 'Feature2')]')",
 [string]$logFileName = "$(Read-Host 'Enter the Log File Name. [e.g. Sample.log]')",
 [string]$action = "$(Read-Host 'Enter [SD] to deploy solutions or [SR] to retract the solution')"
)

Step 2 – Define the Main function.

As many of you know, PowerShell is all about defining functions and these functions can be called later to execute the PowerShell script logic. Main function is heard and sole of this script. Please note that since this PowerShell script would support only if Batch files, log files, and WSPs exists in same directory during deployment process, first step is to retrieve current location of the script. Next step is to create the log file to output the transcript. It would delete the log file, if it already exists.

One of the major tasks of the main function would be loading Microsoft.SharePoint.PowerShell snapin. This PS script will be using many of the SharePoint cmdlets available in Microsoft.SharePoint.PowerShell snapin and it is necessary to load before using the cmdlets. It is also necessary to check whether it’s been already loaded, otherwise this step would throw an error.

Depends on action switch, this function would run deployment or retraction process which will further call other secondary functions for deployment/retraction, activation/deactivation process.

# Defination of main function
function main() {
 # find the current directory
 $currentDirectory = Get-Location
   
 # delete existing logfile
 $logFilePath = "$currentDirectory\$logFileName"
 if (Test-Path $logFilePath)
 {
  Remove-Item $logFilePath
 }
 
 # create new log file and start logging
 Start-Transcript $logFilePath

 # check to ensure Microsoft.SharePoint.PowerShell is loaded
 $snapin = Get-PSSnapin | Where-Object {$_.Name -eq 'Microsoft.SharePoint.Powershell'}
 if ($snapin -eq $null)
 {
  Write-Host "Loading SharePoint Powershell Snapin"
  Add-PSSnapin "Microsoft.SharePoint.Powershell"
 }
 
 # deploy the solution
 if ($action -eq "SD")
 {
  Write-Host "Step 1 - Add Solution Package: " $solutionName -foregroundcolor Green
  AddSolution
  
  Write-Host "Step 2 - Deploy Solution Package: " $solutionName -foregroundcolor Green
  InstallSolution
  
  Write-Host "Step 3 - Timer Job to deploy the Solution Package" -foregroundcolor Green
  Wait4TimerJob

  Write-Host "Step 4 - Activate Site Collection Level Features" -foregroundcolor Green
  ActivateSiteFeatures
  
  Write-Host "Step 5 - Activate Web Level Features" -foregroundcolor Green
  ActivateWebFeatures
 } 
 
 # retract the solution
 if ($action -eq "SR")
 {
  Write-Host "Step 1 - Deactivate Web Level Features" -foregroundcolor Green
  DeactivateWebFeatures
  
  Write-Host "Step 2 - Deactivate Site Collection Level Features" -foregroundcolor Green
  DeactivateSiteFeatures
  
  Write-Host "Step 3 - Uninstall Solution Package: " $solutionName -foregroundcolor Green
  UnInstallSolution
  
  Write-Host "Step 4 - Timer Job to Retract the Solution Package" -foregroundcolor Green
  Wait4TimerJob
  
  Write-Host "Step 5 - Remove Solution Package: " $solutionName -foregroundcolor Green
  RemoveSolution
 }
 
 # stop the logging
 Stop-Transcript
}

Step 3 – Define all the Helper Functions

Define the AddSolution function which would use Add-SPSolution PowerShell command to add the solution to the configuration database. Please note that this step just adds the solution entry in the configuration database. It doesn’t deploy the solution or doesn’t deploy the custom files on the SharePoint Servers yet.

# Add the solution package
# Adds a SharePoint solution package to the farm Solution gallery, It doesn't deploy the uploaded solution yet.
function AddSolution()
{
 $solution = Get-SPSolution | where-object {$_.Name -eq $solutionName}
 if ($solution -eq $null)
 {
  Write-Host "Adding solution package" -foregroundcolor Yellow
  $solutionPath = "$currentDirectory\$solutionName"
  Add-SPSolution -LiteralPath $solutionPath -Confirm:$false
 }
}

Define the InstallSolution function which would use Install-SPSolution to deploy solution to specific web application or all web applications. If solution contains web part, module, event receiver, content type, or any other web application specific SharePoint project item, it would deploy solution to the specific web application otherwise, it would deploy to all the web applications. This step would call Install-SPFeature automatically and adds features entry in the configuration database.IT will also copy all the features and code files the SharePoint Root, copy DLLs into the GAC, and add necessary safe control entries to the Web.Config database.

# Deploy the solution package 
# Deploys an installed SharePoint solution on all the WFE servers in the farm.
# Deploying solution in the farm installs the feature automatically. It installs all the features
# on each server at the farm, web, site collection, and site level but It doesn't activate them.
# Since it provisions the file on each WFE, it is a timer job.
function InstallSolution()
{
 $solution = Get-SPSolution | where-object {$_.Name -eq $solutionName}
 $solutionId = $solution.Id
 if ($solution -ne $null)
 {
  $solutionDeployed = Get-SPSolution -Identity $solutionId | where-object {$_.Deployed -eq "False"}
  if ($solutionDeployed -eq $null)
  {
   if ( $solution.ContainsWebApplicationResource )
   {
    Write-Host "Deploying solution package to web application: " $webAppUrl -foregroundcolor Yellow
    Install-SPSolution -Identity $solutionName -WebApplication $webAppUrl -GACDeployment -Confirm:$false
   }
   else
   {
    Write-Host "Deploying solution package to all web applications" -foregroundcolor Yellow
    Install-SPSolution -Identity $solutionName -GACDeployment -Confirm:$false
   }
  }
 }
}

Define the ActivateSiteFeteatures function which would use Enable-SPFeature PowerShell command to activate all the site collection level features specified in $siteFeatureNames parameter. This would also add feature activation entry in the Content database features table.

# Activate the Site level features
function ActivateSiteFeatures()
{ 
 if ($siteFeatureNames -ne $null)
 {
  $spSite = Get-SPSite $siteUrl  
  foreach($siteFeatureName in $siteFeatureNames)
  {
   Write-Host "Trying to Activate Site Collection Level Feature: " $siteFeatureName -foregroundcolor Yellow
   $siteFeature = Get-SPFeature -site $spSite.url | where-object {$_.displayname -eq $siteFeatureName}
   if ($siteFeature -eq $null)
   {
    Write-Host "Activating Site Level Features at " $spSite.url -foregroundcolor Yellow
    Enable-SPFeature –identity $siteFeatureName -URL $spSite.url -Confirm:$false
   }
  }  
  $spSite.Dispose()
 }
}

Define the ActivateWebFeatures function which would use Enable-SPFeature PowerShell command to activate all the web level features specified in $webFeatureNames parameter in all the webs in specific site collection. This would also add feature activation entry in the Content database features table.

# Activate the Web level features
function ActivateWebFeatures()
{
 if ($webFeatureNames -ne $null)
 {
  $spSite = Get-SPSite $siteUrl
 
  #Cycle through all webs in the collection and activate all the features
  foreach($spWeb in $spSite.AllWebs)
  { 
   foreach($webFeatureName in $webFeatureNames)
   {  
    Write-Host "Trying to Activate Web Level Features: " $webFeatureName -foregroundcolor Yellow
    $webFeature = Get-SPFeature -web $spWeb.url | where-object {$_.displayname -eq $webFeatureName} 
    if ($webFeature -eq $null)
    {
     Write-Host "Activating " $webFeatureName " at " $spWeb.url -foregroundcolor Yellow
     Enable-SPFeature –identity $webFeatureName -URL $spWeb.url -Confirm:$false
    }
   }
  }
  
  $spWeb.Dispose()
  $spSite.Dispose()
 }
}

Define the DeactivateWebFeatures function which would use Disable-SPFeature PowerShell command to deactivate all the web level features specified in $webFeatureNames parameter in all the webs in specific site collection. This would also remove feature activation entry from the Content database features table.

# Deactivate the Web level features
function DeactivateWebFeatures()
{
 if ($webFeatureNames -ne $null)
 {
  $spSite = Get-SPSite $siteUrl
 
  #Cycle through all webs in the collection and deactivate all the features
  foreach($spWeb in $spSite.AllWebs)
  { 
   foreach($webFeatureName in $webFeatureNames)
   {  
    Write-Host "Trying to Deactivate Web Level Features: " $webFeatureName -foregroundcolor Yellow
    $webFeature = Get-SPFeature -web $spWeb.url | where-object {$_.displayname -eq $webFeatureName} 
    if ($webFeature -ne $null)
    {
     Write-Host "Deactivating " $webFeatureName " at " $spWeb.url -foregroundcolor Yellow
     Disable-SPFeature –identity $webFeatureName -URL $spWeb.url -Confirm:$false
    }
   }
  }
  
  $spWeb.Dispose()
  $spSite.Dispose()
 }
}

Define the DeactivateSiteFeatures function which would use Disable-SPFeature PowerShell command to deactivate all the site collection level features specified in $siteFeatureNames parameter. This would also remove feature activation entry from the Content database features table.

# Deactivate the Site level features
function DeactivateSiteFeatures()
{
 if ($siteFeatureNames -ne $null)
 {
  $spSite = Get-SPSite $siteUrl
  foreach($siteFeatureName in $siteFeatureNames)
  {
   Write-Host "Trying to Deactivate Site Collection Level Feature: " $siteFeatureName -foregroundcolor Yellow
   $siteFeature = Get-SPFeature -site $spSite.url | where-object {$_.displayname -eq $siteFeatureName}
   if ($siteFeature -ne $null)
   {
    Write-Host "Deactivating Site Level Features at " $spSite.url -foregroundcolor Yellow
    Disable-SPFeature –identity $siteFeatureName -URL $spSite.url -Confirm:$false
   }  
  } 
  $spSite.Dispose()
 }
}

Define the UnInstallSolution function which would use UnInstall-SPSolution to retract solution from specific web application or all web applications. If solution contains web part, module, event receiver, content type, or any other web application specific SharePoint project item, it would retract solution from the specific web application otherwise, it would retract from all the web applications. This step would call UnInstall-SPFeature automatically and removes features entry from the configuration database. IT will also delete all the features and code files from the SharePoint Root, remove DLLs from the GAC, and remove necessary safe control entries from the Web.Config database.

# Retract the solution package
# Retracts a deployed SharePoint solution from the farm entirely for all web application or given web application.
# This step removes files from all the front-end Web server.
# Please note that retracting solution in the farm uninstalls the feature automatically, if it hasn't uninstalled using UnInstall-SPFeature.
# Since it removes the file on each WFE, it is a timer job.
function UnInstallSolution()
{
 $solution = Get-SPSolution | where-object {$_.Name -eq $solutionName}
 $solutionId = $solution.Id
 if ($solution -ne $null)
 {
  $solutionDeployed = Get-SPSolution -Identity $solutionId | where-object {$_.Deployed -eq "True"}
  if ($solutionDeployed -ne $null)   
  { 
   if ( $solution.ContainsWebApplicationResource )
   {
    Write-Host "Retracting solution package from web application: " $webAppUrl -foregroundcolor Yellow
    UnInstall-SPSolution -Identity $solutionName -WebApplication $webAppUrl -Confirm:$false
   }
   else
   {
    Write-Host "Retracting solution package from all web applications" -foregroundcolor Yellow
    UnInstall-SPSolution -Identity $solutionName -Confirm:$false
   }
  }
 }
}

Define the RemoveSolution function which would use Remove-SPSolution PowerShell command to remove the solution from the configuration database. Please note that this step removes the solution entry from the configuration database and completely cleans up the custom solution deployed on the SharePoint environment. 

# Remove the solution package
# Deletes a SharePoint solution from a farm solution gallery
function RemoveSolution()
{
 $solution = Get-SPSolution | where-object {$_.Name -eq $solutionName}
 if ($solution -ne $null)
 {
  Write-Host "Deleting the solution package" -foregroundcolor Yellow
  Remove-SPSolution $solutionName -Confirm:$false
 }
}

Implement the Wait4TimerJob which will be helper function used after InstallSolution and UnInstallSolution process to ask PowerShell script to wait until solution properly deployed or retracted from all the servers in the farm. It checks timer job status to check whether it’s been completed or not.

# Wait for timer job during deploy and retract
function Wait4TimerJob()
{
 $solution = Get-SPSolution | where-object {$_.Name -eq $solutionName}
 if ($solution -ne $null)
 {
  $counter = 1  
  $maximum = 50  
  $sleeptime = 2 
  
  Write-Host "Waiting to finish soultion timer job"
  while( ($solution.JobExists -eq $true ) -and ( $counter -lt $maximum ) )
  {  
   Write-Host "Please wait..."
   sleep $sleeptime 
   $counter++  
  }
  
  Write-Host "Finished the solution timer job"   
 }
}

Step 4 – Call the Main Function to Execute PowerShell Script Logic

Last step of the PowerShell script is to call the “Main” function defined earlier. This will ask PowerShell engine to run deployment and retraction logic.

#Run the Main Function
main

This concludes the step by step PowerShell script creation process for automated SharePoint solution deployment and retraction process. Next article in this series will provide full version of this PowerShell script and walk through step by step process of how to use this PowerShell script in real world using batch files to automate the process.

Automated SharePoint 2010 Farm Level Solution Deployment and Retraction Process – Part I – Basics

Article Series Content

• Basics of Farm Level SharePoint Deployment and Retraction (This Article)
• Step by Step Building Deployment and Retraction PowerShell Script
• Final Version of Deployment and Retraction PowerShell Script
• Step by Step Using PowerShell Script in Batch Files

If you have been developing custom solutions on SharePoint 2010 platform, having knowledge of how to move your code from developer’s machine to server environment using scripts is mandatory. It’s been best practice for a while to script deployment process while moving code to server environment. It always surprises me seeing developers manually logging on the servers and run PowerShell commands to deploy/retract custom solutions and activate/deactivate features. If you are SharePoint administrators, it is most important that you would ask your development team to standardize the deployment process using scripts.

During SharePoint 2007, STSADM was available to script the custom solutions deployment/retraction process. In SharePoint 2010, Microsoft has introduced PowerShell as preferred technology to script custom solutions deployment/retraction process. This blog series will outline what are the major steps required for farm level custom solution deployment and retraction process for SharePoint 2010, how you can automate the process using PowerShell, and how you can automate the deployment execution using windows batch files.

This is first part of this article series and it provides the overview of the SharePoint 2010 Farm Level Solution deployment/retraction process.

In SharePoint 2010, both Solutions and features can be managed by Browser Interface, PowerShell, or STSADM. You must be farm administrators to manage farm level features, Manage Web permission at the root level site (site collection administrator) to manage site collection features, and Manage Web permission (site owners) to manage site level features.

Following two tables outlines what is supported and commands available at the browser, STSADM, and PowerShell.

Solutions Actions	Central Admin (System Settings-> Farm Management -> Manage Farm Solutions)	PowerShell	STSADM
View Solutions	Yes	Get-SPSolution	-o enumsolutions
Add Solution – Add Solutions to Farm Solutions Gallery	Not Supported	Add-SPSolution	-o addsolution
Deploy Solution – Deploys Solutions to the web applications	Yes	Install-SPSolution	-o deploysolution
Update Solution – Update the deployed solution	Not Supported	Update-SPSolution	-o upgradesolution
Retract Solution – retract solutions from the web applications	Yes	Uninstall-SPSolution	-o retractsolution
Remove Solution – Remove the solution from the farm solutions gallery	Yes	Remove-SPSolution	-o deletesolution

 

Feature Actions	Farm, Application, Site, and Web Level Features Page	PowerShell	STSADM
View Features – View features at the farm, web application, site collection, or site scope	Yes	Get-SPFeature	Not Supported
Install Feature – Install feature at the farm, web application, site collection, or site scope	No	Install-SPFeature	-o installfeature
Activate Feature  – Activate feature at the farm, web application, site collection, or site scope	Yes	Enable-SPFeature	-o activatefeature
Deactivate Feature – Deactivate feature at the farm, web application, site collection, or site scope	Yes	Disable-SPFeature	-o deactivatefeature
Uninstall Feature – Uninstall feature at the farm, web application, site collection, or site scope	No	Uninstall-SPFeature	-o uninstallfeature

 

Typical Order of the Farm Solutions Deployment using the Power Shell Commands

If you haven’t been familiar with the SharePoint Deployment and Retraction cycle, here is the high level diagram depicting the process. Although upgrading solutions and features is not included in visual digaram, Please note that typical steps would be  => Add Solution -> Deploy Solution -> Install Feature -> Activate Feature -> Update Solution -> Upgrade Feature -> Deactivate Feature -> Uninstall Feature -> Retract Solution -> Delete Solution

 

 

Since STSADM has been deprecated (still supported) in favor of PowerShell, this article series will use PowerShell from here on. Here is the high level overview of all the PowerShell commands to deploy/retract solutions, activate/deactivate features, and upgrading solutions and features.

• Add Solution – Add-SPSolution
• Adds a SharePoint solution package to the farm Solution gallery, It doesn’t deploy the uploaded solution yet.
• Add-SPSolution -LiteralPath “c:\Project Initiation Web Parts.wsp”
• Deploy Solution – Install-SPSolution
• Deploys an installed SharePoint solution on all the WFE servers in the farm. This command also installs all the features on each server at the farm, web, site collection, and site level but It doesn’t activate them.
• Since it provisions the file on each WFE, it is a timer job. It may take while to provision the files.
• Install-SPSolution –Identity “Project Initiation Web Parts.wsp” –WebApplication http://sp2010vm -GACDeployment
• Major Parameters
• Identity – Name of the fully trusted solution
• AllWebApplications or Web Application with URL – URL of the web application where solution will be deployed. For Global deployment for all web applications, specify AllWebApplications
• Local – Solution will be deployed on only server where this command is being run
• GACDeployment – To deploy artifacts into GAC
• Time – Schedule the solution deployment at specified time. If this parameter is not specified, solution deployment process would run immediately during next timer job process
• Major File System changes
• Global Assembly Cache – C:\Windows\assembly (e.g. Deploys the DLL into GAC)
• Web Application Directories – C:\inetpub\wwwroot\wss\VirtualDirectories (e.g. Updates the web.config and adds web part safe control entries – C:\inetpub\wwwroot\wss\VirtualDirectories\80)
• SharePoint Root Folder – C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14
• Adds new feature directory and provision the web part files – C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\FEATURES\
• Adds new user controls  – C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\CONTROLTEMPLATES\
•  Install Feature – Install-SPFeature (Optional Step – included in Install-SPSolution)
• Deploying solution in the farm installs the feature automatically. This is preferred method. No need to manually run this command.
• Install-SPFeature <FeatureFolderName>
• Activate Feature – Enable-SPFeature
• Enables installed SharePoint Feature at the given scope. If the feature is a farm feature, no URL is needed. Otherwise, provide the URL where the feature is to be activated (explicit scope is not needed).
• Enable-SPFeature –identity “MyCustomFeatureNameorGUID” -URL http://sp2010vm
• Identity parameter is feature folder name or feature GUID
• Update Solution – Update-SPSolution
• Upgrades a deployed SharePoint solution in the farm. Use this cmdlet only if a new solution contains the same name, same set of files, and features as the deployed solution.
• If files and features are different and feature version is not upgraded, the solution must be retracted and redeployed by using the Uninstall-SPSolution and Install-SPSolution cmdlets, respectively.
• Update-SPSolution -Identity contoso_solution.wsp -LiteralPath c:\contoso_solution_v2.wsp -GACDeployment
• Deactivate Feature – Disable-SPFeature
• Disables installed SharePoint Feature at a given scope. If the scope of the Feature is the farm, the URL is not needed. Otherwise, provide the URL at which this Feature is to be deactivated (explicit scope is not needed).
• Disable-SPFeature –identity “MyCustomFeatureNameorGUID” -URL http://sp2010vm
• Identity parameter is feature folder name or feature GUID
• Uninstall Feature  - UnInstall-SPFeature (Optional Step – included in UnInstall-SPSolution)
• Retracting solution in the farm uninstalls the feature automatically. This is preferred method. No need to manually run this command unless solution retraction process couldn’t uninstall features for some reasons.
• UnInstall-SPFeature <FeatureFolderName>
• Retract Solution – Uninstall-SPSolution
• Retracts a deployed SharePoint solution from the farm entirely for all web application or given web application.  If web application is not specified,  this cmdlet removes files from all the front-end Web server.
• Since it removes the file on each WFE, it is a timer job. It may take while to delete the files.
• Uninstall-SPSolution –Identity “Project Initiation Web Parts.wsp” –WebApplication http://sp2010vm
• Most items that go in the content database are not overwritten or deleted for you when the feature reinstalls or solutions are retracted. Most items that go on the file system are overwritten or deleted during the retraction process. There is exception to this rule – redeploying web parts deletes the web part from the web part gallery and imports the new updated version, if web part deployment conflict resolution property is set to the Automatic. Other options are None to ignore over write and prompt.
• Removes deployed components from the file system
• Deletes the DLL from the GAC – C:\Windows\assembly
• Updates the web.config and removes web part safe control entries – C:\inetpub\wwwroot\wss\VirtualDirectories\80
• Deletes the feature directory – C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\FEATURES\ProjectInitiationToolWebParts_ProjectInit
• Delete Solution – Remove-SPSolution
• Deletes a SharePoint solution from a farm solution gallery
• Don’t deploy the solution when its being removed by this command. It may put the solution in the inconsistent state. Always verify that solution is removed from the farm solution gallery before kicking off the deploy process of same solution.
• Remove-SPSolution -Identity “Project Initiation Web Parts.wsp”

That concludes this overview article. Hopefully you have gained enough knowledge of SharePoint 2010 custom deployment and retraction process. Next article in this series will discuss steps required to build PowerShell script to automate the deployment/retraction solutions process.

SharePoint 2010 Service Applications Quick Reference

SharePoint 2010 Service Application infrastructure is beast. Because Microsoft has built SharePoint Service Application architecture as pluggable, many of the out of the box service applications has been built for different needs with different configurations. Some of the key service applications components like service accounts, backend database, whether it’s available in SharePoint foundation or server product, whether it’s multi-tenant enabled, whether it’s cross-farm enabled are key to understand and will take you long way while designing service applications architecture during initial/ongoing farm configuration.

Additional References

• Microsoft Press – SharePoint 2010 Administrator’s Pocket Consultant Book, Chapter 2, Table 2-1
• Wrox’s Professional SharePoint 2010 Administration Book, Chapter 7, Table 7-1
• Technet Poster – Topologies for SharePoint Server 2010 – http://go.microsoft.com/fwlink/p/?LinkID=167089

I have put together a high level grid to map all the major components of service applications for quick reference. I would encourage you to take this reference table and map it to your specific farm infrastructure. Enjoy.

Service Application	SKU	Recommended Location of Server	Stores Data? If Yes, Provide DB Name	Cross Farm Capable?Publish-able?	Windows Service?	Multi-Tenant Aware?	Is IIS Web Application Associated?	Is IIS WCF Service Application Associated?Managed from Central Admin?
Access Service	Enterprise	Application	Cache	No	No	No	No	Yes
Application Registry Service	Foundation	Application	DB (Application Registry)		No		No	Yes (only deployed using PowerShell or FCW)
Business Data Connectivity Service	Foundation	Application	DB(BDC)	Yes	No	Yes	No	Yes
Central Administration	Foundation	Application	DB(SharePoint_AdminContent)	N/A	No	N/A	Web App, SharePoint Central Administration v4, Runs as Farm Admin account	N/A
Claims to Windows Token Service	Foundation	Application	No	No	C2WTS, Installed as part of WIF. Disabled by default. Enabled by this Service. Runs as the local system user.		No	No
Document Conversions Launcher Service		Application					No	No
Document Conversions Load Balancer Service		Application					No	No
Excel Service	Enterprise	Application	Cache	No	No	No	No	Yes
Lotus Notes Connector		Application (Index)					No	Yes (Search)
Managed Metadata Service	Standard	Application	DB (Managed Metadata)	Yes	No	Yes	No	Yes
Microsoft SharePoint Foundation Incoming Email	Foundation	Web or Application					No	No
Microsoft SharePoint Foundation Subscriptions Settings Service	Foundation	Web or Application	DB (Subscription Settings)	No		Yes (PowerShell Only)	No	Yes (only deployed using PowerShell)
Microsoft SharePoint Foundation User Code Service	Foundation	Web or Application		No	SPUserCodeV4 – Runs under SP Service Account		No	No
Microsoft SharePoint Foundation Web Application	Foundation	All Web Servers, Stop on Application Servers		No			No	No
Microsoft SharePoint Foundation Workflow Timer Service	Foundation	Web		No			No	No
PerformancePoint Service	Enterprise	Application	DB (Performance Point)	Yes	No	No	No	Yes
PowerPoint Service	Office Web Apps	Application	Cache	No	No	No	No	Yes
Search Query and Site Settings Service	Standard	Application (All Query Servers), Load balances queries across all Query servers				Yes	No	Yes (Search)
Secure Store Service	Standard	Application	DB (Secure Store)	Yes	No	Yes	No	Yes
SharePoint Foundation Help Search Service	Foundation	On Foundation Farm, Search Server.On Server Farm, Application, Start this Service only on one computer.	DB (Help)	No	SPSearch4 – Runs Under Local Service user		No	No
SharePoint Server Search	Standard	Automatically Configured to run on Appropriate Servers in the farm.This Service can’t be started or stopped from the Central Admin.  Search components are provisioned from the Search Admin Page.	DB (Search Admin, Crawl, Property)	Yes	OSearch14 – Runs under SP Service Account	Yes	No	Yes (Search)
State Service	Standard	Application	DB (State)	No	No	No	No	Yes (only deployed using PowerShell or FCW)
Usage and Health Data Collection Service	Foundation	Application	DB (WSS_Logging)	No	No	No	No	Yes (only deployed using PowerShell or FCW)
User Profile	Standard	Application	DB (Profile, Social Tagging, Synchronization)	Yes, Requires both farms to be in trusted AD domain.	No	Yes	No	Yes
User Profile Synchronization Service	Standard	Application	No	No	FIMService FIMSynchronizationService Windows Services are provisioned by the User Profile Sync Service. Runs as the Farm Admin account.	??	No	No
Visio Graphic	Enterprise	Application	Blob Cache	No	No	No	No	Yes
Web Analytics Data Processing Service and Web Analytics Web Service	Enterprise	Application	DB (Reporting, Staging)	Yes	WebAnalyticsService -Runs under SP Service Account	No	No	Yes (Web Analytics)
Word Automation	Standard	Application	DB (Word Automation)	No	No	Yes	No	Yes
Word Viewing	Office Web Apps	Application	Cache	No	No	No	No	Yes