This clinical trial is for men and women with advanced urothelial cancer. The purpose of this study is to research the effects (both good and bad) of two investigational drugs called erdafitinib and JNJ-63723283. All patients enrolled in this study will receive either erdafitinib alone or in combination with JNJ-63723283 depending on the time of enrollment. Erdafitinib is taken by mouth every day. JNJ-63723283 is taken via IV infusion once every 2 weeks. Participants will continue on treatment as long as they are responding to therapy and not experiencing unacceptable side effects.