Unlike convex optimization landspaces, they counter non-convex ones such as Neural networks have multiple local/global minima’s which results in same performance. Even with same starting point, we could end up at different minima’s depending on the optimizer.
Optimizers decide the effective expressivity of the models, induce inductive bias in the final model.