Hello Babu,
AX does not allow you setting up item groups and subgroups that built a hierarchy.
If you need to record transactions for a specific "sub-group" on ledger accounts you should possibly setup this specific subgroup as a separate Dynamics AX item group. In other words, technically you do not setup a "subgroup" but an ordinary item group.
Let's say that you have "food" as your general item group that you use for posting most of your transactions and that you need to record transactions for vegatables on different ledger accounts then you can establish an item group denominated "food-vegetables" or something alike.
Rather than setting up your item postings based on item groups you can also define them for specific items. Yet, this will only work if you have a limited amount of sub-items that need a special accounting treatment. Otherwise, setup will take much time and needs to be changed often.
Please note that item groups are primarily used for accounting purposes - not for reporting purposes. If your concern is differentiating the different items that you have for reporting reasons only - that is you do not need transactions on different ledger accounts - then you should use another field - e.g. a financial dimension or another field - than the item group for this purpose.
Hope it helps,
Ludwig