391 comments

danlitt
> ethical approaches to model development, how humans interact with AI and debate over machine consciousness

So, pseudo-philosophical mumbo jumbo. There is of course no-one responsible for the ethical framework of the company's actions, because it is a company, and therefore amoral at best and immoral at worst.

greggsy
I hate these companies as much as the next guy but this perspective is childish at best, and ignorant at worst.
danlitt OC
Which perspective? I said two things.
kaon_2
I am not sure in what neighbourhood you live, but in my village there are hundreds of businesses with strong morals. From the local baker, to the electrician and the construction company. Many keep employees past their prime, they all invest heavily in local community events and sports. The owners don't do this because of money, but because in their view a fulfilled life is one where you serve your community. And yes they want to become rich and make money, but that is not all.

But it appears that is not how your world works. By default any company owner is a homo-sapiens, but a machine that only wants money. Tragically, your cynicism is creating the very world you hate.

Your neighborhood examples are on an incomparably smaller scale than Meta et al.

It’s much easier to disregard the human cost of decisions when you are separated from said humans by thousands of miles & billions of dollars.

danlitt OC
Many people have strong morals, and might resist profit-maximising behaviours, but they will be outcompeted and replaced by less scrupulous people. There is actually no sharp difference between a person and a one-person-company, but: to the degree a company is really a company and not just a person, it is amoral. If you disagree, please point to a moral backstop that would prevent any company from taking immoral acts. A few exist, but they are invariably resisted by the most powerful companies. Sorry to be the bearer of bad news.
The people and procedures inside are the backstop.
pferde
can be the backstop. The word "can" is doing some extremely heavy lifting here, against corporate practices which dilute and spread guilt and responsibility for immoral decisions so thin, nobody in the decision chain even realizes something immoral has been decided.
Have you ever been in committees approving new products or perhaps certain transactions? These chains aren't necessarily that long. But "can", sure, things could fail etc. and some places are better than others at it.
Is this the same argument as "please point to a moral backstop that would prevent any person from taking immoral acts"? We usually hope that people will be moral, but can't enforce this.
pferde
My point was that in big corporations, things are stacked against people wanting to act morally.
danlitt
How many leadership meetings have you been in where some profitable approach was refused on an ethical basis? (rather than the basis that behaving ethically is actually more profitable, which is totally different)

By contrast, how many conversations have you personally had with people working at a company believing a profitable action was firmly unethical, but felt forced to behave that way due to "business realities" or some other whitewashing?

Nursing has been around for a long time and they have not yet had the ethics in their field crushed by some of the biggest forces in our corporate-friendly system. High school teachers too. I could probably come up with a dozen more given half an hour to think about it. Don't give up so easily. People fought hard to get us here.
You're comparing small business to a corporation? Right.
But what happens when one of those businesses in your village strikes it big and becomes a multi-national baking good company. They hire a lot of MBAs, take investor money and then need to stay profitable to meet those obligations. Add to the fact that the people now running the business live nowhere near the village. Will they still make the most ethical decisions about keeping employees past their prime?, paying them a fair wage?

History shows this is not the case.

I'm happy for you to live in a place where the local economy is so vibrant. This is what capitalism should have been. Instead we got WallMart.

I know. The local newspaper believes the same thing. And the local events are all sponsored by businesses. Who could question all of this evidence?

> But it appears that is not how your world works. By default any company owner is a homo-sapiens, but a machine that only wants money. Tragically, your cynicism is creating the very world you hate.

They said companies are amoral at best.

Bet none of them have a chief ethicist tho.
I know people like that, you are not wrong.

The problem is that corporations completely subvert any individual morality that might indicate that generating money is the only moral thing to do. Typically in small businesses. I comaider small businesses owners in general are a lot closer to labor than to capital owners in general terms, especially when their labor is still a large part of what keeps the business going.

Corporations are not that, those are hyper optimized for generating profits, all else be damned. Preferably if all else is damned.

Zigurd
This is how Ayn Rand's pseudo philosophical mambo jumbo continues to poison tech.
So, concerning ethical approaches to model development, there are many things worth looking into: copyright is one, environmental cost is another, and whether there are any additional concerns worth taking into account. The list of these concerns can then be used for decision making, and we can ask things like "given what we want to build, can does picking a location that gives us lower environmental cost add so much time or cost that we can't go for that path?", things like that are't crrazy.

When it comes to how humans interact with AI we can ask things like "would it be better if people could detect AI output?" "how can we alleviate skill loss from excessive AI reliance?" Things like this can decide how the models behave. Imagine if taking concerns like this seriously leads to models that are easy to learn from. At the moment people seem to be moving away from that so as to make distillation difficult, worsening this concern.

I'm not going to try to talk about machine consciousness, but these other things are definitely important questions where someone digging into them could make models interact better with society.

[flagged]
Well, economics in general doesn't, but yes.

There isn't much ethical merit in building a wall around a city with guard towers, attack dogs, and anti-personnel mines to keep the citizens in so that they can contribute to your economic vision, for example.

You mean like in East Berlin?
BRD and DDR were in a quite particular competition. The DDR contained less than 20% of the German people, and West Germany would happily have absorbed them all.

The capitalistic world state system was only completed in the last 50 or so years and was of course seconded by the Stalin machine. Its purpose is in the first instance to keep working populations in place, which is why all the states agree to it as they form. The system holds solid where capital moves (as did 'communism' and Comecon) but the peoples don't.

It is only when you think of individual states and not the system that you can fail to notice that we get the same result if they all try to keep their citizens in, and if they all try to keep the others' citizens out.

Exactly East Berlin, actually.

But that's just one extreme example. States based on Marxism-Leninism and Maoism in the 20th century often required things like internal passports to travel within a republic's borders. If the state didn't want you visiting family in another part of the country, you weren't visiting family. That still applies to some extent in a few modern single-party socialist states.

My larger point is, a lack of ethics consumes most economic systems, because you can't easily gauge ethics like you can quarterly profits or the output increase brought about by the central committee's five-year plan. Humans tend toward math over empathy in such situations.

tclancy
Eh, I would say it doesn’t have them as axioms the concept is based on, but ethics are modelable in economics and often appear. Remember economics is the study of choices/ preferences given a certain set of inputs, not about how money works. That’s just the easiest second effect signal to use. Game theory regularly models how ethics affect and are affected in situations.

All of that said, no idea what this person was doing. Probably involved a red stapler.

pcblues
Surprisingly it does. It allows private property and the ability to progress with effort and mutual respect. Got a better alternative?
> Before working at OpenAI, she was Meta’s chief ethicist
So more of an ethicist-for-pay than for the challenge of it.
jofzar
It's silicon valley, literally everything is for pay or be homeless
That's everywhere, there are homeless people everywhere. The difference is Silicon Valley feels so guilty about it's wealth it refuses to do the normal things that societies do to prevent people from living like animals because 'rights' or something, so they let drug addicts live in their common spaces without treatment. This normally isn't tolerated in a functioning society where people are pressured into behaviors that are less self destructive.

When someone isn't competent to make decisions for themselves it's ok to pressure them into engaging in behavior that will lead them to get to a condition where they can start to make decisions for themselves. In Silicon Valley they have let their guilt be leveraged by a corrupt government system that lets grifters steal billions of dollars per year in 'homelessness funding'. Those same grifters fund political groups and bribe politicians. Anyone that questions it is accused of wanting to murder homeless people.

No, homeless people are not everywhere.

Homelessness is created by high rents. High rents are created by high local productivity. The people who are productive enough to make it over the rent-bar are housed, and their marginal productivity above the rent-bar serves to raise the rent-bar (talk to any landlord about how they set rent prices: it's the local wages, stupid). This then pushes a greater proportion of people below the rent-bar, and then they are homeless.

There are few places in the world more productive per square foot than Silicon Valley, ergo one must be extremely productive in order to remain housed there, ergo it has an extremely bad homelessness problem.

That's why people making $200k+ (an extremely productive person by national standards) need public housing assistance in Silicon Valley (https://www.reuters.com/article/world/palo-alto-considers-su...). Which of course actually just... raises the rent again...

The charities may be full of grift and whatnot, but you can remove them entirely and you will not solve the fundamental problem.

shafyy
> High rents are created by high local productivity.

Wrong. High rents have many reasons and also good policies to get them down and have affordable housing for more people, also in otherwise expensive and popular cities.

OP has the demand part of the equation if supply remains fixed. The reason supply remains fixed is the US as a whole made policy changes in the 1970s that slowed down construction, specifically zoning laws and environmental laws swept the country. The 2008 recession also cancelled what little construction remained ongoing for a long time.

When you combine that with spiking wealth in Silicon Valley, you’ve got the perfect storm in Silicon Valley. With OpenAI and Anthropic both based in SF, it’s going to be worse than ever once they IPO.

The equilibrium for new housing construction will always be "just enough to keep it expensive by local standards [i.e. by local wages]." That's especially true with rising labor, material, and capital costs.

It was actually the post-war housing boom that was abnormal, fueled by aggressive federal funding of suburban buildout. US municipalities are now broadly doomed to bankruptcy trying to maintain sprawling infrastructure that was never cost-effective to create in the first place.

Look at Austin TX. Influx of new building, prices stabilized, and now construction is dropping off a cliff. No one can afford to build into a stable or potentially falling local market. Construction will start again when local productivity grows enough for people to bid the prices back up again.

Nope.

Rents cannot (at meaningful scale) exceed local productivity, or else there will be widespread vacancies and landlords will reduce their rents.

Rents cannot (at meaningful scale) fall below local productivity, or else landlords will simply raise their rents.

Go talk to a landlord about how they site prices.

Step 1. Look up income of your target demographic

Step 2. Eat 30% of their income

Can you show me a place with high local productivity and low rent?

Can you show me a place with high rent and low local productivity (that's not e.g. a vacation destination for people with high productivity elsewhere)?

You can certainly artificially suppress prices with rent control. But that just gets fed back into the system in the form of reduced supply, so it ends up raising rents on the non-controlled housing stock anyway.

> Can you show me a place with high local productivity and low rent?

Put the following prompt into your favourite AI and enjoy the reading:

"How do rents relate to productivity in the area? How does Vienna compare to San Francisco?"

Vienna? World famous for publicly-funded housing construction?

Try this one: "Create a simple analysis approximating how much higher rents would be in Vienna if not for the aggressive public subsidization of supply"

In any case, Vienna has a good system! But it's not that it avoids high productivity → high rent, it's that it splits rent into subsidy costs and rent.

In other words, the amount of subsidy + rent required will grow with local productivity.

My point is: Your post reeked by defeatist 'nothing can be done against market forces'.

Yes, it can be done, and if we want to have a reasonable future, it should be done.

So you were responding to an assumption that's not actually in my comment?

Vienna's solution is fine but has escalating costs and isn't necessary. Land Value Tax is better.

kbelder
Assume you have an area with a certain productivity and rent. Population and housing are growing at a certain rate.

Now make a law that makes construction cheaper. Or make a law that makes construction more expensive. What happens to rent and mortgages, even though productivity hasn't changed at all?

Productivity (or, rather, wealth) of the occupants is a very strong factor, but it is still just one of multiple factors.

You think you can make a law that sets the price of housing above local wages and stays there?

No, you actually cannot do this in any lasting way.

The price of land would fall to the level required to make it construction viable again. Or the city would be abandoned, which is not generally the outcome since cities have so much shared capital investment that cannot be moved elsewhere.

> Rents cannot (at meaningful scale) exceed local productivity

Manipulating the real estate market via direct economic manipulation, collusion, or regulatory capture is so common that the entire concept of manipulating the market to earn higher profits is called https://en.wikipedia.org/wiki/Rent-seeking.

It's not just that it can, it's that it does so often it's taught in Econ 101. That you don't know this implies you have never taken an economics class or that you forgot what you were taught, and in the meantime certainly have not spent any real effort in learning about economic theory. I think you read a lot of Ayn Rand and right wing libertarian popular books about the free market and how it's perfect. Stop posting confident nonsense.

No, what's happening here is you misunderstood the quoted text, silly.

The claim is not that rents are always at their optimal or most efficient level - they're not! The claim is that you cannot (at meaningful scale) charge people who earn $1,000/month in income $2,000/month in rent to live in a place. They will have to move.

Did you take any arithmetic classes?

And any case, you should stop mind-reading. You're bad at it. I'm not a Randian nor a right-winger nor a libertarian nor free-market idealist.

Lets say I own 50 properties and I want to charge $2000 a month, and right now I can't because the people willing to rent my properties only make $1000 a month. I can bribe politicians to not allow any new construction of properties which rent between $2000 and $4000 per month. Then I wait 10 years. Now I can charge $2000 a month because I'm not renting to the people that made $1000 a month 10 years ago, I'm renting to the people who have $2000 a month to pay for rent but can't find any other units. This is called rent-seeking. You should read about it.

Your mistake was thinking that I, the corrupt landlord who pays bribes, cares about a macro analysis of every all the properties, or that some analysis about the current distribution of incomes and possible rents will stop me from doing exactly what you say is impossible. I don't own every property in the city, especially the ones that aren't built because I capture zoning, so arguments about average GDP and average rent and whatever else do not stop me. Your analysis is useless.

bulbar
That's not how it works at all. You can't pressure people into becoming a productive member of society.

What you can do when it already happened is kill them or make them die in some ways, jail them or try to support and/or treat them.

I hope it goes without saying that letting them just die or killing or jailing them is not an option in a western society.

Prevention is much more favorable of course. For example, don't let people become homeless easily. But even then, a significant amount will become homeless.

Until you accept that they didn't 'easily become homeless' the problem will continue. They used drugs and spiraled, usually over the course of many months or years, forced all their friends to turn their backs on them, got kicked out of any living situations they had access to because of drug/alcohol use or theft to support the same and finally ended up on the street.

On the street they have access to almost endless support, at least in Silicon Valley, but much of that support is contingent on things like not openly using heroin or not repeatedly getting so intoxicated that they defecate or vomit on themselves, so they refuse it.

I don't care how liberal you are, no society is functioning when it allows people to get on BART and pee on themselves and defecate on the cloth seats and walk up and down the aisles dripping feces and urine on the floor. That is allowed and nothing is done about it for fear of upsetting homeless 'rights' groups who are funded by the tax dollars meant to help homeless people.

Perhaps you aren't aware, but only about a third of homeless people are drug addicts, from the surveys I've seen.
Here is how you get to 1/3:

1. Widen the definition of 'homeless' to cover people that nobody would consider homeless in every day language. Include anyone who is staying with a friend, living in corporate housing, staying with a family member, etc. Include anyone you possibly can that is unlikely to be a drug addict.

2. Narrow the definition of homeless. Don't consider people that almost everyone does consider homeless. Make sure to target the subgroups of homeless people that are very likely to use drugs.

3. Narrow the definition of drug use. Make it as hard as possible to count someone as having used drugs. Raise the evidentiary bar so high that even when you have evidence they are a drug addict it doesn't qualify them as a "drug addict" by the studies definition. Bury that definition so it's technically there but you have to go through the study carefully to find it.

Lets pick a study in a big important journal and check it for the above:

How about CASPEH: "Illicit Substance Use and Treatment Access Among Adults Experiencing Homelessness," JAMA, 2025

JAMA is a very prestigious journal:

Who counts as a substance user: Only meth, nonprescription opioids, cocaine/crack count as addictions. Cannabis, alcohol, benzos, hallucinogens, inhalants are excluded.

Someone who has a valid prescription to opiates, even if they are being abused, cannot be methodologically counted as a drug user. THIS INCLUDES METHADONE.

What counts as addict: "Regular" = 3+ times per week. A twice-weekly meth user is 'occasional use'. They might take a gram and stay up for 3 days twice a week, but they are 'occasional user' and don't count in the 37% addict number quoted by this paper.

What counts as addict: Last 6 months only. If you admit you used fentanyl every day for the last 10 years but claim you got clean 181 days ago, you are not a drug user according to this study. If they have been diverted from heroin onto methadone 181 days ago you do not count.

How did they check? They asked each person face to face whether they used drugs and which ones and how often and just took their word for it.

The real numbers from the raw data the study used:

41.6% of homeless self reported 2x per week or more hard drug use before they became homeless. an additional 23.2% claimed they started regular use after becoming homeless.

That's about 65% already from their own data, 2x what the headline pretends.

Excluding high use subgroups: to be included in the data, the homeless person had to prove they were giving informed consent. Then they additionally had to be willing and able to spend 45 minutes or so filling out the survey. The people with psychosis or who were intoxicated at the time of the interview were excluded. How many people have you walked past and they were incoherent, unconscious, or high? They were ignored in this study.

Homeless people are a better way to keep cowardly minions (I mean, valuable employees) in line than cops. The homeless are an important part of capitalist power structures. They are a visible reminder of the consequences for not meeting quarterly goals.
No, it's actually very unusual to have tons of homeless people next to spectacular wealth, California is a global outlier in this phenomenon.

There are many, many places in the world that are much poorer and don't have homeless encampments all over the place.

I literally said that.
You said "there are homeless people everywhere", which is technically true but omits quantity, and especially quantity compared to wealth.

I'm not as positive as you are about what makes California (and Seattle) create/attract so many, but it's definitely an outlier compared to "everywhere".

Yes, if you didn't understand my point: there are homeless people in every society, but most do not have as many as silicon valley because they pressure them into accepting the help necessary to restore their ability to make rational decisions. Rational people do not choose to live outside on the sidewalk like rats, people who are drug addicted and who have untreated mental illness do.
I'd just be really skeptical of thinking I have the true understanding and solution to this issue.

Like, do you know many addicts or seriously mentally ill? Have you tried pressuring them into being better? How did it work out?

All it takes to be homeless is not being able to afford housing, drugs or mental illness are not required. You really think all it takes is "choosing" and housing will appear for you?
dgellow
Also, drug consumption and mental disorders is very often a result of being homeless. Try having bad sleep in your comfortable home for a few days in a row and see how quickly your mental stability deteriorates and how many bad decisions you start to take (anyone who has insomnia periods will attest)
This is not substantiated by data or even a plausible theory. The kinds of mental disorders that lead to homelessness are not something you can "catch" from your environment.
dgellow
You mean like PTSD? Something that is literally a result of your environment?
esseph
I'm not as positive as you are about what makes California (and Seattle) create/attract

The weather is pretty nice, much more stable than in the east coast.

Rubber stamp for hire?
Tade0
Or someone to assess the impact of all those anti-human ideas shape rotators come up with.
onion2k
I'd accept a few hundred grand to reply to emails with "No, the torment nexus is a net benefit.."
dgellow
They finally found a way to automate that role with GPT!
gotta be more verbose if you want the CEO to think you're worth it. Try this:

"Although the torment nexus may torment us in the short term, over the long term there will be fewer people to torment as we improve the capitalization of human's torment so that fewer people will be available to torment duing to the reduction in overall torment-forces globally, and thus, with less people being tormented, surely that will increase the over all torment reduction of the nexus. Win-win: less tormented people means less torment!

Arch-TK
It adds up...
sph
That’s exactly the situation they found themselves in, and you are admitting you have no more spine and moral compass than them.

Or are you telling us in the hope someone makes you such an offer?

pants2
I'd think an ethicist at Meta/OpenAI involves extremely challenging mental gymnastics
Severance in real life
Any ethicist for a corporation is clearly a PR move, right?

Like, a pure researcher for a corporation might have some genuine overlap and shared interests where breakthroughs in the research benefit both parties, but an ethicist?? For a legal construct that's defined by and bound to sociopathically pursue profit? Come on now.

>Any ethicist for a corporation is clearly a PR move, right?

I think a lot of it depends on the size of the company. Startup? Sure, maybe it's a real thing.

Gigantic company like a member of FAANG or MANGOS? absolutely a PR move.

andor
Companies are not bound to sociopathically pursue profit. It would be in their long-term interest to care about stakeholders outside the boardroom, and until a certain person with tiny hands started a war on anything remotely "woke", the market rewarded ESG metrics.
Your point was true prior to 1980 or so, but it's a cultural average. We've had multiple generations of business leaders now trained on seeing the corporate legal entity as only caring about shareholder value and being proud of it and even perversely principled about it.
andor
It's true that this is happening a lot. And I think this is one of the reasons why ethicists still want to get involved in these companies: it should be easier to make a company do the right thing from the inside, why else would they hire for these positions? They join with idealism and probably don't get very far in terms of real impact.
> They join with idealism and probably don't get very far in terms of real impact.

I can't disagree with that, reminds me of the quote from the movie Rounders: If you've been at the table for 30 minutes and you don't know who the sucker is, you ARE the sucker.

It depends on the kind of company, and the context. An ethicist at a company doing drug/medical research probably mostly oversees that the company's research practices are legal and safe (and that participants give informed consent).

At typical corporation, something like a Chief Ethics and Compliance Officer (CECO) would be largely focused on legal compliance - making sure company personnel aren't engaged in bribery, fraud, insider trading etc. The value of this role (and the medical example) is to prevent lawsuits and/or prosecution.

For a company like Meta, an ethicist is likely concerned with the inherent ethical concerns of running a social media company, which impacts the mental health of its users, many of whom are minors. For AI companies, the ethics of releasing models that are capable of doing dangerous things. In name, the job probably involves giving recommendations about how to ethically implement features in the platform to minimize harm. In practice, the job probably accomplishes very little.

nilamo
Just because the ethicist says something is a bad idea, doesn't mean anyone else is obligated to make a change.

ie: blaming her for anything instead of the decision makers is... really weird?

No influence, same toil? One must imagine her happy.
nilamo
Why must I imagine that? We're assuming it's a position where you're screaming pointlessly into the void, and never accomplish anything. That sounds miserable, I'd leave, too.
One Must Imagine Sisyphus Happy is the concluding sentence of Albert Camus' essay The Myth of Sisyphus. It parallels her situation.
fHr
lmao
ulfw
What a joke really. The 'position' itself and the companies who pretend to care about ethics
yreg
Where is she going next? Nestle?
Philip Morris.
dgellow
ExxonMobile (energy division)
Straight to Cocomelon, catch them young'
Palantir
Oh no, if only the world had more ethicists on staff. If only Nazi Germany had someone with a thorough education in ethics on the payroll!

It's performative. She got paid more to shill for a different massive amoral mega-corporation. Don't be stupid.

No ethics left at OpenAi huh? Lolol
their only ethicist left? gasp!

now they are exactly like every other company that has zero ethicists on staff and expect the employees to weigh the ethical ramifications of their work.

jdiff
I think we can look upon the collective works of the corporations of the world and agree that the employees do not do that, nor are they expected to.
What companies do you think expect their employees to weigh the ethical ramifications of their work?

I’d argue a lot of them expect the opposite. To suppress the ethical concerns.

Even in recent memories you had Mark Zuckerberg telling his employees to use pirated sources. And executives at fossil fuel companies have to act unethically.

Food testing? Hospitals? Childcare? Pharmacists? How about "clergy"? I could go on and on.

Do you believe that no one in management in these fields pushes the importance of ethics?

I think that individuals are capable of making ethical choices. I think that relying on individuals within for-profit businesses to make ethical decisions for the entire company is absolutely insane.

Food-testing: There's literally a parasite going around right now. Look at the response from some of the for profit businesses that was allowed to happen.

Hospitals: It is not ethical to deny patients care based on their insurance.

Clergy: The cover up of child sexual abuse scandals.

Childcare: lots of examples of sub-par care given to save money.

[flagged]
weberer
This was the first post on HN to make me actually laugh out loud.
the only shocking thing to me is that they still had even a single one.
d--b
Think she's been replaced by an LLM?