You either reject H0 or do not reject H0. You never say something is correct or incorrect because you can never be 100% certain. What you are really saying when you reject H0 is that based on the data it is so unlikely that the null hypothesis is right that you are going to assume it is not right.
If you use statistics to make decisions you will not always make the right decision, but you will be right more often than if you do not use statistics.