The Clarets boss felt his side showed some vulnerabilities in allowing the Saints back into the game on the south coast after Chris Wood and Matej Vydra had netted inside half-an-hour.
However, Stuart Armstrong pulled a goal back for the Saints just three minutes later before Danny Ings punished Ben Mee's error to restore parity before the break.
"It's frustrating, it's one of those that got away from us from our own lack of awareness," said Dyche. "We were very good for the first half-an-hour and controlled as much of an away performance as you can.
"We had done the basics very well for a large part of our performance, our two centre forwards were causing trouble, we were playing well in the in-betweens, opening up the pitch and defended well.
"We looked a threat, scored two goals, but then they get back in it from nowhere. We conceded a soft one and then for the second we've given a goal away.
"It's frustrating, but that's football in the Premier League. It's never over and we showed a bit of naivity. It is unusual for us to do that.
"We just didn't do the basics. Defensively we got a little bit complacent, we went onto the back foot a little bit and they got their first goal out of nothing really, and then we've given them a second with an individual error."
Nathan Redmond completed the turnaround after the break when Theo Walcott lifted the ball towards the back post, but Dyche was bemused as to why the visitors weren't awarded a second penalty late on.
Burnley's first goal of the game came from the spot in the 12th minute when - after a delay while VAR checked the incident - Andre Marriner called play back to penalise Kyle Walker-Peters' challenge on Erik Pieters.
It was only the Clarets' second spot kick of the campaign - a number only matched by three other sides (West Ham, West Brom, Leeds United) in the Premier League.
But Dyche couldn't believe that Jan Bednarek's shirt pull on Wood - who forced a save from Fraser Forster with a header from Dwight McNeil's cross - wasn't picked up on.
He said: "We got a penalty today, but they tried not to give us it. Our record of penalties is terrible, I know it's the worst in the Premier League by a long shot.
"They tried not to give us the first one and then Woody gets impeded for the second. Ryan Bertrand handles the ball and there's no penalty, so we do struggle.
"I'm a big fan of VAR and I think it will get better, but those are the ones that you scratch your head at. You have to give a penalty for me because he clearly has hold of his shirt.
"If that's in the middle of the pitch then it's given as a free kick. Our record of penalties in the Premier League is one of the worst!"