User-agent: MJ12bot Disallow: / User-agent: * Allow: /$ Allow: /spa/*/public/apply Allow: /spa/public/apply Allow: /External/OJCustomer3/AssignmentView.aspx Allow: /Upload/MediaBank/ Disallow: /