13GB avaliable space disabled in “df” command, but shows 50GB+ free space in “About this Mac”->“Storage”
Donghuas-MacBook-Air:~ donghua$ df -H
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk1s1 121G 106G 13G 89% 1485339 9223372036853290468 0% /
My mac enabled with time machine features, these extra space caused by localsnapshots.
Donghuas-MacBook-Air:~ donghua$ sudo tmutil listlocalsnapshots /
com.apple.TimeMachine.2019-02-18-222558
com.apple.TimeMachine.2019-03-26-112425
com.apple.TimeMachine.2019-03-26-122807
com.apple.TimeMachine.2019-03-26-142211
com.apple.TimeMachine.2019-03-26-152326
com.apple.TimeMachine.2019-03-26-165332
com.apple.TimeMachine.2019-03-26-175311
com.apple.TimeMachine.2019-03-26-205645
com.apple.TimeMachine.2019-03-26-215449
Manual remove some snapshots. (or plug in portable disk to run backup)
Donghuas-MacBook-Air:~ donghua$ sudo tmutil deletelocalsnapshots 2019-02-18-222558
Deleted local snapshot '2019-02-18-222558'
Space released
Donghuas-MacBook-Air:~ donghua$ df -h
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk1s1 113Gi 82Gi 29Gi 74% 1485491 9223372036853290316 0% /
One more alternative is to set below envronment variables before start the pyspark command:
ReplyDeleteexport PYSPARK_DRIVER_PYTHON=/path/to/anacona3/bin/jupyter
export PYSPARK_DRIVER_PYTHON_OPTS="notebook --NotebookApp.open_browser=False --NotebookApp.ip='*' --NotebookApp.port=8888"