User-agent: * Disallow: /wp-admin/ Disallow: /wp-login Disallow: *.pdf # Block pdf files. Non-standard but works for major search engines. Allow: /wp-admin/admin-ajax.php