More twitter tomfoolery

December 16, 2017

I wrote a Go program to install blocks from a file of Twitter IDs. It’s not on github yet because my development sandbox looks more like a development catbox, and I can’t clean it up too much because the program’s running right now and Twitter’s rate limited so it’ll be a while (at 5 blocks per second, about 10 days). Recall that my goal is to completely remove fascists and griefers from my Twitter feed, and from any conversation that I happen to be in — they shouldn’t even notice the opportunity to respond, never mind wasting anyone’s time with their crap.

At least as important as the program is the list of IDs to block.
It’s 4 million lines long, sorted from most-to-least-desirable-to-block order, so this is the only way to share it. I did some by-hand sampling, and the first 10% really are notably more obnoxious than the last 10%, so I may not run this all the way to completion.

package main

import (


var digits = regexp.MustCompile("[0-9]+")
var runtime = time.Now().Format(time.RFC3339)

type ConsumerAndAppKeysAndSecrets struct {
	ConKey, ConSecret, AppToken, AppSecret string

var caksFile = ".twitter/ids"

/* The .twitter/ids file contains four lines with string values
   obtained from the twitter developer api.

ConKey = "..."
ConSecret = "..."
AppToken = "..."
AppSecret = "..."

   For an App token and secret, you need to create an app here: https://apps.twitter.com/app/new
   This will then give you the option to create a consumer key and secret.
   (This useful information cribbed from
   https://stackoverflow.com/questions/1808855/getting-new-twitter-api-consumer-and-secret-keys )


func main() {
	caks := &ConsumerAndAppKeysAndSecrets{}
	blob, err := ioutil.ReadFile(caksFile)
	if err != nil {
		fmt.Printf("There was an error opening or reading file %s: %v\n", caksFile, err)

	err = toml.Unmarshal(blob, caks)
	if err != nil {
		fmt.Printf("There was an error unmarshalling contents of %s: %v\n", caksFile, err)

	api := anaconda.NewTwitterApi(caks.AppToken, caks.AppSecret)
	fmt.Println("Credentials = ", *api.Credentials)

	a := os.Args
	if len(a)  0 && i%1000 == 0 {
			flush(users, i)
			users = users[:0]
		user, err := api.BlockUserId(int64(u), url.Values{})
		if err != nil {
			errst := err.Error()
			if !strings.Contains(errst, "User not found.") {
				fmt.Printf("i=%d, u=%d, err=%v\n", i, u, err)
				flush(users, i)
		} else {
		users = append(users, user)

	flush(users, len(wlids))


type tomlWantsStruct struct {
	users []anaconda.User

func flush(users []anaconda.User, ending int) {
	buf := new(bytes.Buffer)
	if err := json.NewEncoder(buf).Encode(users); err != nil {
		fmt.Printf("There was an error encoding users: %v\n", err)
	fname := fmt.Sprintf("Blocked-%s-%08d", runtime, ending)
	err := ioutil.WriteFile(fname, buf.Bytes(), 0666)
	if err != nil {
		fmt.Printf("There was an error writing %v: %v\n", fname, err)

func readFileAsUint64s(filename string) (uids []uint64, err error) {
	var b []byte
	b, err = ioutil.ReadFile(filename)
	if err != nil {
	bids := bytes.Split(b, []byte("\n"))
	uids = make([]uint64, 0)
	for i, bid := range bids {
		s := digits.FindString(string(bid))
		if s == "" {
		var uid uint64
		uid, err = strconv.ParseUint(s, 10, 64)
		if err != nil {
			fmt.Printf("Failure to parse line %d of %s\n", i, filename)
		uids = append(uids, uid)

New Twitter Algorithms

November 5, 2017

My Twitter block list got unmanageably large, and blocktogether.org was not even able to remove blocks at any sort of a reasonable rate to help me fix it. So, I used my employer’s mighty-fine search engine to look for any Go packages for the Twitter API, and found Anaconda.
Read the rest of this entry »

The last time I did this, I had figures through 2011.
Now I have 2012, 2014 and 2105 (2013 seems to be missing).
Now in a Google spreadsheet, so you can look at the numbers directly and poke at the links if you want to see where the numbers came from.

In words — since 2009, each gallon of gasoline or diesel is taxed between 40 and 50 cents too low even if the only purpose of that tax is to pay for road construction and maintenance. Any other taxes (carbon, pollution, noise, congestion, health care) would be on top of that. This also does not include the maintenance or construction that we ought to be doing; this is just what is spent.

Totaled over all the fuel sold, each year since 2009 the annual shortfall totals somewhere between 75 and 100 billion dollars.

Read the rest of this entry »

Twitter algorithms

July 2, 2017

These are my rules for making Twitter more useful.

My goal, on Twitter, is a combination of finding fun and interesting stuff and to expose myself to (certain) other points of view. At work we have training on bias, unconscious and otherwise, and on techniques for reducing it and countering it. One of the instructors mentioned that you can’t just wish unconscious bias away; apparently repeated exposure to normalizing examples is required, but it takes time (this is yet another disturbing/annoying way that our brains resemble neural nets for machine learning; in this light, unconscious bias is just the result of a lifelong biased training set.)

Read the rest of this entry »

It’s a little depressing to look at how many hard it is to get all the different factions of the Democratic Party excited about helping each other. I wonder a bit if this is a case of scarcity pushing people towards fighting over scraps, and I wonder how much this is a case of Russians/Republicans using the internet to sow left-wing dissent.

At minimum, people ought to accept that each others’ problems are worthy. Is there really any question that blacks get a raw deal in this country? Or that people who are openly gay or trans are discriminated against? Or that women don’t get promotions and pay commensurate with their skills, productivity, etc? Or that unions are necessary in order to give workers an equal footing in negotiations over pay, hours, benefits, and worker safety? Or that many forms of pollution lead to statistically early death? Lack of an adequate social safety net is clearly a problem, and clearly one that can be solved, because countries that are less wealthy do a better job than we do — notably, they deliver life expectancy and lower infant mortality for less money per capita. They can afford it, so can we. Climate change? It’s happening. Slowly, but steadily, and it’s going to continue for decades-to-centuries after we finally decide to take it seriously; the only question is how fast it’s changing when enough of us finally get alarmed enough to really act. Education? College is stupidly, fantastically expensive, and to the extent this is Baumol’s Cost Disease, we should just subsidize it (other poorer countries manage to do this) and to the extent that it isn’t we should drive prices down by properly supporting public universities. Etc. These are all problems, and the Republican Party is on the wrong side of all of these issues. We shouldn’t pick just one, we should not be put off because we think labor is important but we’re a little nervous about the gays, or focus only on racism to the exclusion of college costs — there’s nothing wrong with wanting it all, we can have it all, and all of us deserve to have these problems addressed. There’s no mutual incompatibility between any of these issues.

And be a little more skeptical, say, when someone on Fox News tries to tell you that anyone who’s LGBTQ is a threat to the women and children. We’ve done plenty to make life unpleasant for people who aren’t “normal”; if someone’s out of the closet and you notice them, they must feel very strongly about it to put up with the social crap, and must have been truly miserable in the closet. This has nothing to do with your children, and everything with them wanting to live happier lives. Anyone who tells you otherwise is trying to con you into being mean to other people for no reason at all; ignore them, they’re evil.

Or, similarly, that someone might trot out some bogus statistics to try to make white people nervous about “black crime”. Some of these stats are flat lies, in other cases the data has been tortured into confessing things that aren’t true. In practice, most people are non-violent, most people are law-abiding (well, except for traffic laws, which everyone breaks very often, and traffic violence is actually a big deal). Don’t take the bait, anyone trying to convince you that blacks are a Big Crime Risk is just plain evil, ignore them, change the channel, turn off the radio. They’re trying to turn you into a racist and create dissent on the left.

There are bullshit artists trying to sow doubt about health care, too. One dishonest clown keeps trying to claim that Medicaid is worse than no health care at all, because people on Medicaid (as a population) are sicker than people who aren’t, never mind that if you’re poor and sick you’re much more motivated to sign up for Medicaid than if you’re merely poor, in which case that might seem like more of a hassle than it’s worth. This is what passes for serious statistical analysis on the right; these guys are sad, lying clowns, don’t let their obvious bullshit make you doubt the worth of providing health care.

And so on. There’s probably better examples but I’m a cis het white guy 1%er descended (father’s side) from a family with strong ties to Dartmouth, clearly I’m a traitor to my gender, race, ancestors, etc, it’s a wonder I get any of this right. The main theme is to not let one left-wing cause be split from another, and anytime you catch someone trying to do that, think about why. I honestly wonder how many of the alleged “hard-core Bernie-bros” that get noticed on the internet now are actually left wing or even American; disinformation is a real thing, and sowing dissent is a standard tactic. I supported Bernie, I sent him money, I like (or liked) his politics. But when he didn’t win the nomination, we’re done, support the nominee, got to stay focused on outcomes. I have several friends who did the same. Ask yourself *why* someone on the left would now be interested in prolonging the primary contest after we lost the general election. It makes no sense; the Republicans are uniformly terrible for everything Bernie Sanders has supported over the years, the Democrats are uniformly better, and we tried plenty hard in the primaries and Bernie didn’t make the cut. If we don’t unite, all of us, we lose ground.

Been meaning to write something, always too distracted to “do a good job”, as if getting nothing written was a good job. So….

Just now read a Copenhagenize article on bikes and trains saying something I had believed, but had no data to support. They have data. They also point out by example yet another way we do bikes wrong here in the US.
Read the rest of this entry »

Charitable plans

November 12, 2016

SPLC, NAACP, CAIR and/or ICNA, Planned Parenthood, Lambda Legal, Trans Lifeline.

and ACLU, EFF, National Popular Vote.

Any other suggestions? I think I’m a little light on defending rights of immigrants.

Oops — As JF points out in email, ADL.

I’d also like to fund organizations doing voter registration work, especially in swing states, especially in states where Republicans narrowly control legislatures and/or executive. We need to reduce the amount of gerrymandering in this country, we need representation in the House of Representatives that more nearly reflects the popular sentiment, and we need to ensure that we are well safe from crazy constitutional amendments (a constitutionally mandated balanced budget would be a macroeconomic disaster; recessions would turn into depressions).

I realize I am setting myself up for a deluge of please-help-our-worthy-cause solicitations, both electronic and paper. We get those already, plan is to set up a spreadsheet, and just give once a year, every year.