I need some help for the robots.txt file, to avoid a duplicate content problem with my url rewriting... But I'm lost

I've found that code for example in a tutorial:
But this is not allowed, a "*" after a "disallow" instruction...

why not? surely robots.txt just tells bots what to do?

Yep it tells the bots what to do and not to do

But in the code, a "*" can't after a "disallow"

different bots use slightly different rules. how do you know you cant use a * after a dissalow?

It's a general rule ( http://www.robotstxt.org )

Note also that regular expression are not supported in either the User-agent or Disallow lines. The '*' in the User-agent field is a special value meaning "any robot". Specifically, you cannot have lines like "Disallow: /tmp/*" or "Disallow: *.gif".

whole article: http://www.robotstxt.org/wc/exclusion-admin.html

ah gotcha. is there another character, something like ? you can use?

Presumably the * is a 'wildcard'...which doesn't work with this board....in which case the 'wildcard' for this board must be something else....
....he says from a complete position of ignorance

cheers

James

you kinda had teh right idea, but not entirely right. never mind eh