Hi sanjay.1816,
You will always see the checkmark in the legal entity form because the hierarchy needs to be published with a datetime stamp. Meaning that you can change and modify an existing hierarchy. However, a hierarchy that has been published cannot be deleted from UI. I have not tried deleting an older version of a once published Org.Hierarchy but would imagine that this can cause some difficulties because those hierarchies have a lot of integrations - for example with Workflows and the Management Reporter. So, if you delete them from backend, please carefully test all of your processes, Workflows and Reports to ensure that they still work properly.
Best regards,
Ludwig