Duplicates are automatically removed by our system. You can work with a datastream containing unique articles.