There are many reasons why Democrats find themselves on the wrong end of a gerrymandering armageddon.

There’s John Roberts and the US supreme court, who pretended partisan gerrymandering is just politics as usual, left voters naked to extreme power grabs, and failed the nation when voters most needed the courts’ protection.

That 5-4 decision in 2019 would have been different if not for Mitch McConnell, who prevented Democrats from filling an open seat on the court in 2016, and preserved it for the Republican party and Neil Gorsuch.

But perhaps the most important reason is the brilliant 2010 Republican strategy called Redmap – short for the Redistricting Majority Project – which left Republicans in charge of drawing lines for four times as many congressional seats as Democrats, and close to 70% of state legislatures nationwide.

Just a few years earlier, jubilant Democrats had celebrated Barack Obama’s 2008 victory and dreamed that America’s changing demographics would lead to a decade of triumphs and a new permanent majority. It did not work out that way – because they fell asleep on redistricting.

The following election, Republicans captured the approximately 110 state legislative districts they needed to dominate congressional redistricting. They held the House in 2012 despite winning 1.4 million fewer votes than Democrats, and haven’t looked back since. Democrats are still trying to catch up – and now, even as the party insists it’s going to fight back against Republican gerrymandering, remain hamstrung by snoozing more than a dozen years ago.

How could a party with such a genuine demographic edge get out-organized, out-strategized and out-energized in election after election? How could no one have seen the looming redistricting nightmare? How did they do nothing about this when they controlled a trifecta in Washington with a filibuster-proof majority in the Senate?

Turns out some people did issue warnings. When I wrote my book Ratf**ked: Why Your Vote Doesn’t Count, the definitive history of how Republicans gerrymandered the nation, I went in search of the wise men and would-be Paul Reveres, the people who knew all about the importance of redistricting, but whose shouts vanished into a black hole of complacency, overconfidence and unimaginative thinking.

Had Democratic leadership listened to Steve Israel, John Tanner and Martin Frost then, all of this could have been avoided.

After the Republican rout of 2010, Israel, a New York congressman then in his sixth term representing suburban Long Island, took over as chairman of the Democratic congressional campaign committee. If Washington is a city filled with unpleasant jobs, Israel stepped into one of the most hopeless. The DCCC chair serves a two-year sentence as a party road warrior, raising money, barnstorming chicken dinners and county barbecues, and most importantly, trying to recruit congressional candidates who might actually be able to flip a district. A successful term pole-vaults a politician into leadership. But swing districts are few – and few ambitious mayors or state senators want to sacrifice careers and endure those barbecues themselves only to lose an unwinnable race. So the chairman bounces from one Hampton Inn to the next, marshalling every drop of persuasion.

Israel spent four years doing this. His second marriage collapsed. The late nights, the loneliness, the flight delays all seemed so unbearable that the only relief came from writing a novel on his iPhone that was a vicious satire of Washington ridiculousness.

You can imagine why all that travel might have seemed worth it. The 2010 spanking meant that basic competence would look good by comparison. Also, 2012 brought a presidential cycle, and Democrats actually turn out to vote in presidential years. Sometimes that enthusiasm even trickles down-ballot and helps elect Democrats to Congress. But that was before it became clear how the Republicans had used gerrymandering to push their 2010 advantage into a durable and lasting majority. As he studied the new districts and criss-crossed the country, Israel may have been the first national Democrat to realize how ratfucked his party was – and how long it would last.

“What shocked me when I first came into the DCCC was when I learned that the expansive battlefield that I thought I would have at my discretion was actually a pretty small map,” Israel told me. “There are a couple dozen competitive districts, maybe … You can have the best recruit, the best candidate, the best fundraising. But if you have an uncompetitive district, there’s no path.

“I mean, the math proves it,” he says, and you hear the anguish of every night at a chair hotel bar with a burger and a bad Syrah. “Look, we won 1.4 million more votes than they did in 2012 and we only picked up eight seats. That tells you that this whole thing was jury-rigged in order to stop Democrats from playing in competitive districts. It worked brilliantly for them. I’m just sorry we didn’t figure that out in 2008.”

As Israel sees it, that’s the year when Democrats really screwed up. He thinks the party should have been thinking ahead then to redistricting and down-ballot races. Instead, they planned for nothing. Redistricting, he says, never seemed to cross the mind of Democratic leadership. It was, he says, “a catastrophic strategic mistake”. In 2006 and 2008, Democrats “won districts that we had no business winning. But then we started losing state legislatures and governors across America – and that’s what destroyed us in 2010 and 2012. Had we devoted resources to protecting Democrats in state houses across America, the Republicans still would have won the majority in 2010. But we would have had a seat at the table in redistricting and we might have been able to take it away from them in 2012.

“The DNC,” he says, shaking his head, “they just whistled past the graveyard. I don’t understand why.”

Republicans, he says, “have always been better than Democrats at playing the long game. And they played the long game in two fundamental ways. Number one, on the judicial side. They realized they had to stock courts across the country with partisan Republican judges and they did it. The second long game was on redistricting. The center of gravity wasn’t an immediate majority in the House. It was rebuilding the infrastructure in courts and state houses across the country so when they got the majority back they could stay in it for a long, long time.”

Israel walks me to his office door. “I’ll tell you one thing,” he says. “This wouldn’t have happened if Martin Frost was still here.”

Frost, a Texas Democrat who served from 1979 until 2005, and Tanner, a Tennessee Democrat who held office from 1989 until 2011, were the two Democrats in previous Congresses who really understood the long-term ramifications of redistricting and agitated, usually alone, for action. Both are long gone from the Capitol, but when I hunted them down for my book I found them where I half expected: steps from K Street, along the Washington DC legal and lobbying corridor where former pols cash in on years of connections and experience.

Tanner, then the vice-chair of Prime Policy Group, had a cushy corner office with a putting green, a cushy landing for an 11-term Democrat from Tennessee. Exhausted by partisanship, and well aware that even his reputation for bipartisanship would not save him when Tennessee Republicans redrew congressional lines after 2010, Tanner chose not to seek re-election. And so Republican gerrymandering claimed one Democrat who had repeatedly tilted at a then lonely windmill: redistricting.

As his fellow moderate Blue Dog Democrats disappeared, white southern Democrats went extinct, and congressional partisanship began to harden, Tanner was moved to take action. In three successive Congresses, under both Democratic and Republican control, he tried to put a stop to partisan gerrymandering. He proposed national standards that removed the power to draw distinct lines from state legislatures and handed it to commissions. His plan also prohibited redrawing lines more than once in a decade, which would have prevented the gerrymandering armageddon now under way. This was not an issue that made the otherwise garrulous Tanner a lot of friends. Neither Democrats nor Republicans wanted anything to do with it.

“Here?” Tanner says of Washington. He pushes at a cup of coffee. “Ha! They’re drawing their own districts. I had many members come up to me and say, ‘What are you doing?’ They have deals. ‘Don’t come around here fucking with the maps. I won’t fool with your map if you don’t fool with mine.’”

Tanner first introduced his plan in 2005, when Republicans ran the House. Tanner knew it would be an uphill battle, and indeed, his bill never earned as much as a committee hearing. When Democrats took back the chamber after the 2006 election, he thought he might convince his leadership to listen. He flagged down the House speaker, Nancy Pelosi, and the majority leader, Steny Hoyer, who ignored him and wriggled free.

“I told them, if you don’t do this, all the population growth is under Republican control. The only stronghold left for Democrats is cities … They didn’t want anything to do with it.”

Tanner remembers Pelosi saying: “We’ll take a look at it.” But he couldn’t get a hearing on his bill in 2007 or 2009 either. Partisan warriors, he suggested, never really want to reform the process. They might fight to take away the other side’s advantage, but never, ever do they want to risk their own.

Might Democratic elders regret ignoring him now, as leaders of a permanent minority? Tanner snorts again. “It’s just not something anyone wants to take up. I went through (redistricting) three times. There’s a lot of power connected to that system.”

Tanner says that far more than 300 seats are responsive only to the most partisan elements.

“We can’t even do the small problems now, let alone the big ones,” he says. “These guys are trapped in this system where the only threat is from their base in a primary … No one will do what they all know has to be done to keep the country from going adrift. Is that because of redistricting? Hell, yes.”

Tanner speaks with appealingly frank disgust for a man whose living was long based on his relationships with these same pols. “Democracy? The people’s will? It doesn’t matter,” he says. “That’s redistricting, too. The average citizen is a pawn. Without the protection of a fairly drawn district, the citizen is a pawn of billionaires who use the map of the country as a checkerboard to play politics on.”

Hidden behind owlish glasses, Frost doesn’t look the part of an aggressive warrior, but he is the last hardened Democratic street fighter to serve in the House. When we spoke, he escorted me into a conference room with a well-appointed cookie tray and explained how he had learned the importance of redistricting after Texas gained three seats in Congress after the 1990 census.

In the 1990s and 2000s, Frost watched as Republicans sought maps that packed as many voters of color as possible into one district – knowing that doing so would create whiter and more Republican seats in the surrounding areas. Sometimes they even worked together with Black Democrats. Frost represented the Dallas-Fort Worth area and it became clear that one of these new seats would be a majority Black district, which had the potential to cut into his base. Frost wanted to stay in Congress, and wanted white and Black Democrats to work together to create districts that would benefit both. As he wrote in his book The Partisan Divide: “The survival of white Southern Democrats would be determined by how many Black voters were left over for their districts after the new majority Black seats were created.

“So I started asking the question, ‘Who is doing redistricting for the Democratic party? I wanted to talk to that person. I was stunned by the answer. No one.”

Texas by the 1980s was trending red, but Democrats still controlled the legislature and the governor’s office, and therefore redistricting. They came up with a plan that added three new districts whose voters were largely people of color without dismantling the bases of the white incumbents. Frost calls it “a classic example of what could be done when all members of a state Democratic delegation work together for the common good.” Texas Democrats extended their advantage in the US House from 19-8 to 21-9. The Frost gerrymander held until Republicans took the state house in 2002, and the House majority leader, Tom DeLay, pushed the legislature into a mid-decade redistricting plan, much as is happening now.

The problem for Democrats is that despite these repeated lessons in the importance of line-drawing, no one continued Frost’s work after the DeLay map knocked him out of Congress. “For a while, we fought them to a standstill because we had good legal talent and technical help. Then we just got overcome on the political side.”

How is this possible? I ask. “I’m not the right one to ask that question to,” he demurs, but says he thinks about it all the time. He has concluded that the party’s coastal and white leadership simply doesn’t understand what it’s like to run for office as a Democrat outside of Pelosi’s San Francisco. “Leaders in the Democratic party come from safe, white districts. So they don’t worry about these things, because nothing can be done to them. You can’t do anything to Nancy Pelosi’s district.

“White northern leaders don’t think of this the same way that white southern politicians think about it. We instinctively understand the problem, but white liberals didn’t really focus on this very much. They said, ‘Well, everything’s fine. We’ll just continue what we’re doing’ and didn’t make this a priority. I argued for 20 or 30 years about the importance of paying attention to state legislatures, but I couldn’t get enough people in the party to really embrace that. The Republicans understood that and had a strategy. We didn’t.”

Frost even became chairman of the DCCC after the 1994 Newt Gingrich rout, but, like Israel later, could never convince anyone else in power to take redistricting seriously. I tell him what Israel said, that this wouldn’t have happened had anyone listened to Frost, and he gives a quick nod that suggests he agrees.

“No one else in the party cared about this or understood how important it was, for whatever reason.” The Republicans not only got it, but knocked out the one Democrat who did too. “Maybe I wouldn’t have been able to change history, but we sure as hell would have gone down fighting.

“It didn’t have to be. If the Democrats had put the same type of emphasis on redistricting that the Republicans did, there might have been a different outcome. Could have been. Should have been. We’ll never know.”

• David Daley is the author of Antidemocratic: Inside the Right’s 50-Year Plot to Control American Elections as well as Ratf**ked: Why Your Vote Doesn’t Count