Rihanna and Chris Brown to be 'X Factor' Judges?

I know that with LA Reid departing X Factor after this season that the rumors will start flying about judges for next season and the first one is something sure to be controversial.
Apparently Simon Cowell wants both Rihanna and Chris Brown to be judges for the show. Yeah, sounds ridiculous, but it sure would bring ratings to the show. Can you imagine them having to work with each other everyday and the smirk on Simon's face if he managed to land them?
That would of course mean that Britney Spears or Demi Lovato would also be shown the door. I think of all the judges on the show, Demi would want to keep the gig most of all because it is a place for her to make some money and keep in the spotlight and I'm not sure she would be able to do so based solely on her music . although, I will enjoy "Skyscraper" any day. -->