SharePoint 2010 Search Shikhar Thapa Senior Consultant 11/05/2010 Agenda • • • • • • Scale-out Architecture End-user Enhancements Social Search Extensibility Enhancements Deployment and Manageability FAST Search for SharePoint Products for Every Customer Need Quick, easy, powerful search (for free!) Basic search Complete intranet search Intranet-wide search People and expertise search High-end search delivered through SharePoint Visual experiences, extreme adaptability and advanced content processing Powerful platform to build on • 360° customer views • Research portals • Reputation analysis • … Search Center - UI for users to issue queries and interact with results Query Servers - Accept query requests from users and return results OpenSearc h Source Query Object Model Query Federation - Return results from non- Query Servers SharePoint Indexes Indexing - Extract information from items to enable Index Partition efficient matching Index Partition - Subset of the overall index Indexer Crawling - Traverse URL space to record items in search catalog Crawler Connectors - Know how to process different content sources Content Content Content Agenda • • • • • • Scale-out Architecture End-user Enhancements Social Search Extensibility Enhancements Deployment and Manageability FAST Search for SharePoint MOSS 2007 Search Scale-out “The whole index” Query “Bottleneck” Indexer Query + Query “Bottleneck” “Single point of failure” SharePoint Server 2010 Search Scale-outMultiple Indexers Stateless Crawlers Crawl Distribution Admin Database Admin Component Query Query Query Mirroring Index Partitioning Multiple Prop DBs “The whole index” Admin Database + Admin Component Query Query “Bottleneck” “Single point of failure” Indexer Indexer Crawler Crawler “Bottleneck” Crawler Crawler Content Distribution • Crawl Distribution – In-built load balancer distributes hosts to crawl databases – Crawlers crawl content that is covered by crawl database – Default configuration can be overwritten using host distribution rules • E.g. purchasing a new connector • Query Distribution – Low query latency if all index partitions equal in size – Distribution by hash of documentId – Crawlers partition indexed data and propagate to query servers Agenda • • • • • • Scale-out Architecture End-user Enhancements Social Search Extensibility Enhancements Deployment and Manageability FAST Search for SharePoint End-User UI • Out-of-box refinement – Refine over key results properties – Metadata, taxonomy and social tags based results refinement – Easy to extend over custom properties • One-stop Search Center – Scopes, web parts, best bets, top answers , advanced search – Query federation brings together results from all over - native support for OpenSearch • Core search experience – – – – Improved did you mean suggestions New pre-query and post related query suggestions “View in browser” link (for most office docs) Improved query syntax End-User UI • Improved relevance ranking – New ingredients for Relevance – Improved low-noise snippets in summaries • Enhanced multi-lingual support – Automatic detection of language of many document types and part of documents – Compound word handling – Improved ranking of documents in multilingual collections • New form factors – Mobile search from Smartphone browsers – Desktop search integration in Windows 7 Search Center • Site Template • Tabbed Interface • Multiple Entrypoints • Keywords, Definitions, and Best Bets • Refinement Panel • Extensible Search Usability Features Refine by query type, and many other pivots Tabbed Interface Query Suggestion. One Point of Entry Best Bets/Did you Mean Improved result layout and hithighlighting New Query Syntax • Support for Boolean operators for FreeText queries and property queries – (“Search” OR “SharePoint Search”) AND (title:“keyword syntax” OR title:“query syntax”) • Prefix matching support for keywords and properties – Bench* jobtitle:Scien* • Improved operator support for property restrictions – =, >, <, <=, >= – Can create range refinements Demonstration Agenda • • • • • • Scale-out Architecture End-user Enhancements Social Search Extensibility Enhancements Deployment and Manageability FAST Search for SharePoint Social Search • • • • • Entry from Core Results Fuzzy name search Self-Search Expertise/Profile Search Refinements Overview of the social search system Managed Metadata Service Application (Taxonomy) User Profile Service Application Search Service Application Connect with People and Expertise Refine by query type, and many other pivots Phonetic and nickname matching Sort by relevance, name or social distance Vanity search Recently authored content Improved result layout and hithighlighting Demonstration Agenda • • • • • • Scale-out Architecture End-user Enhancements Social Search Extensibility Enhancements Deployment and Manageability FAST Search for SharePoint Common Framework, APIs, Development Tools Richer Capabilities enable search-driven applications Browser-Based Extensibility Options • Search Web Parts – Search Box / Advanced Search – Search Summary – Search Statistics – Search Paging – Search Actions – Search Best Bets / High Confidence Results – Search Results – Search Refinement • Search Tab Pages Search Input Web Parts • Search Box – Scope options – Additional query terms – Advanced Search Page – Search Results Page – Query Suggestion • Advanced Search – Input types – Properties (metadata) – Search Results Page Search Utility Web Parts • Search Summary – “Did you mean” – “You searched for” • Search Statistics • Search Paging – Next / Prev Text – Next / Prev Images • Search Action Links – Determine what links to display Search Best Bets / High Confidence Results • • • • • • Same web part, different configuration XSLT for display Show Keywords? Show Best Bets? Show High Confidence Matches Best Bet Limit Search Core Results • • • • • • • Location Results per Page Enable removal of duplicates Enable stemming Selected Columns Configurable XSLT Scope Search Tabs • Based off of lists in Search Center Site – Tabs in Search Pages – Tabs in Search Results • Process: – Create Entry Page – Create Results Page – Configure Web Parts on Entry and Results Page – Add Entry Page to Tabs in Search Pages – Add Results Page to Tabs in Search Results Search Extensibility: End-User UI • Change web part properties – no code – Modify XSLT – Modify config XML • Refinement panel – control metadata available for refinement • Advanced search – control metadata available for advanced search queries • Extend OOB web parts programmatically – All OOB web parts are public – Extend web parts to change default behavior • E.g. Extend core results web part to show result previews • Extend Search Center with vertical experiences – All web parts communicate through public interface (SharedQueryManager object) Bringing it all together… Search Extensibility - Federation • Query Federation – Bring external search results into the SharePoint experience – Based on Open Search standard – Exposed through UI + public federation object model Connector Architecture Search user queries Search page points to Profile Page or custom page to show search result Search results page – results based on metadata Search Engine/Index Protocol Handlers Search BDC Runtime BDC Profile Pages or External Lists Fetch data from repository Execution call routing through BDC Connector using BDC model file WCF Database s .NET BDC Connector Custom BDC Connector Demonstration Agenda • • • • • • Scale-out Architecture End-user Enhancements Social Search Extensibility Enhancements Deployment and Manageability FAST Search for SharePoint Richer Manageability • Consolidated administration UI dashboard • Automated service password management through “managed accounts” • PowerShell support for scripted administration • Built-in system health monitoring, support for SCOM monitoring and alerting • Built-in and extensible search analytics reporting Other Engine Enhancements • Support for regular expressions in Crawl Rules • Native support for crawling case sensitive repositories • Ability to prioritize Content Sources so as to distribute crawler resources • New ‘Crawl Policy’ to define how crawler treats error conditions • Low indexing downtime Search Backups Demonstration Agenda • • • • • • End-user Enhancements Social Search Scale-out Architecture Extensibility Enhancements Deployment and Manageability FAST Search for SharePoint Architecture Best High-End Content Search based on FAST ESP Common query and federation APIs FAST Search Common connectors and content APIs for SharePoint Content search SharePoint Server 2010 End Users People search Unique Social Search from SharePoint Server 2010 Content + Profiles Visual, Conversational Search Visual Best Bets Sorting on any property Refinement with counts on any property Scrolling PowerPoint Previews Thumbnails Similarity Search Beyond Just SharePoint Search Content Processing and Schema • • • Extracted document attributes reported as Crawled Properties Crawled Properties mapped to Managed Properties Characteristics are defined for Managed Properties, e.g. – – – – • Refiners Sorting Queryable Type Report discovered crawled properties Definition and mapping done via UI or Powershell – Not via Central Admin Admin UI Schema CmdLets Custom Client Schema Object Model Update configuration Schema Service (hosted in IIS) Persistence Property backend configserver Update Tools bliss psctrl Alert pipeline of updated schema Document Processing Pipeline PropertiesMapper PropertiesReporter Demonstration Site Collection Scopes • Similar interface to SSP Scope Management • Organize by Display Group • Same flexibility for rules Scope Visibility • Define Display Groups • Groups Control Visibility on Search Web Parts Search Visibility • Control if a site shows up in search results • Sites are already security trimmed • Example: Search Center Set to “No” Keywords & Best Bets • Use query reporting to help define keywords • Use keywords to provide access to content inaccessible to the crawler SharePoint Server 2010 Search Major improvements over MOSS 2007 Area New features and benefits Social Connect with people and expertise MOSS 2007 SharePoint 2010 • Address-book style name lookup with phonetic matching • Improved expertise matching - mined e-mail inbox data • Social behavior improves search; search improves mySites Refinement Find information faster • Metadata-based refiners provide drill-down • Related query suggestions show popular queries Relevance Provide great search results • New ingredients, more experience -> better OOB search • Better wordbreakers in more languages • Social Distance, tagging, click-through influence relevance Platform Industrial Strength Search at High Scale • Index partitioning -> sub-second latency at 100M items • Full failover support • Connector and BDC enhancements and tooling support TCO Deploy and Operate with less effort • Virtualization and native 64-bit simplify deployment • Powershell, Admin web parts, SCOM provide easier admin Extensibility Create powerful applications based on search • Web parts, query OMs are public -> easy to modify • Enhanced query syntax References: http://www.mssharepointconference.com/Pages/default.aspx http://www.microsoft.com/downloads/details.aspx?FamilyID=751fa0d1-356c-40029c60-d539896c66ce&displaylang=en Presentation Material: <MNSPUG> slide link Contact: Shikhar Thapa (email@example.com) Q&A A key element to your success.