This is a hard question because it highly depends on your whole financial situation and is also psychological.
Some elements you may consider:
Purely on a financial side, it depends on your risk tolerance and opportunity cost. How much would you earn if you kept the money? If you have 100% of your portfolio in equities, and you invest everything in the S&P, purely based on CAPM your expected return should be around 8.4 % (10y treasury at about 4, and risk premium at 4.2). If you have different market assumptions or a different risk tolerance and thus portfolio allocation, your expected return will be different. So, based on that, you can see what is financially optimal.
But there is also another important consideration, and this is where your assets matter a lot. Imagine you have 60k saved, and your loans are 50k. Of your assets, the first 20k are your emergency fund, and are barely earning interest. The next 60 are maybe in a safe ish portfolio. Overall, those 60k would be expected to earn let’s say 4 or 5% overall. Ot would seen financially optimal to pay back a 6% loan. However, this would leave you with no debt and only 10k in assets. That is, at risk of having to take on new debt if you have an unexpected expense, possibly at an unfavorable rate.
If in a different example you have 500k in assets and 50k of debt, the situation is different, all else equal. So you can see where this is not just based on interest. You don’t want to pay back a 6% loan only to find yourself in a situation where you have to resort to credit card debt because you spent all the cash paying back the loan.
Tldr: on a purely financial optimization side, it may or may not be optimal to pay back the loan based on your portfolio allocation and market expectations. On a behavioral side, a less financially optimal plan may still be desirable depending on other objectives such as keeping your safety net