So let me try to re-iterate:
1. Accounting currency = X (USD)
2. You have invoice in currency Y (JPY)
3. payment in accounting currency (based of what they consider the exchange rate)
4. settlement date = payment date
Based on the above,
It looks like at that point the amount they calculated in USD does not conform to your own system's exchange rate
You can confirm that by looking at the invoice value in USD at the settlement date, the amount in USD will be different then what you received in USD.
As a result when settling the payment with the invoice, the invoice remains partially open.
In addition AX calculates and posts FrX gains/losses for the partial amount settled.
That's by best guess based on the information you provided.
To avoid having this issue in the future when selecting the invoice in the "settlement" form, select "full settlement" option. That would "force" the payment to fully close the invoice, and move all the differences into the realized gain/loss posting.
For existing transactions, you can use the write off functionality to record it into an expense account.