Oil prices were choppy near $100 a barrel, while the yield on the benchmark 10-year Treasury note US10YT=RR hovered around 4.94% after US President Donald Trump warned in a speech to the UN General Assembly that he could "annihilate the Islamic Republic" if no deal is reached to end the Middle East conflict.
Investors also awaited a US-China summit on Thursday, with traders speculating the trade truce with Beijing could be extended and talks could touch on AI regulation.
"What people are looking for is at least the start of a conversation that maybe you can come up with some sort of global framework as opposed to fighting each other ..." said Joe Saluzzi, head of equity market structure research at Themis Trading.
The S&P 500 ended essentially flat at 7,764.64 points.
The Nasdaq gained 0.45% to 27,244.28 points, its second straight record high close.
The Dow Jones Industrial Average declined 0.36% to 51,863.69 points.
Volume was heavy on US exchanges, with 17.9 billion shares traded, compared to an average of 16.3 billion shares over the previous 20 sessions.
Six of the 11 S&P 500 sector indexes declined, led lower by financials .SPSY, down 1.68%, followed by a 0.8% loss in communication services .SPLRCL.
The S&P 500 was less than 0.5% below its August 13 record high close.
JPMorgan Chase JPM.N and Wells Fargo WFC.N both lost more than 3%.
Boston Federal Reserve President Susan Collins wrote she supported last week’s interest rate increase, citing inflation risks.
New York Fed President John Williams said the Fed's framework for controlling short-term rates has been effective in supporting financial markets.
Traders see a 53% chance the Fed will increase interest rates by at least 25 basis points in October, the CME Group's FedWatch Tool showed.
The S&P 500 saw about the same number of stocks climb as decline.
The S&P 500 posted nine new highs and 16 new lows; the Nasdaq recorded 48 new highs and 110 new lows.