Deep Learning: Difference between revisions
(7 intermediate revisions by the same user not shown) | |||
Line 49: | Line 49: | ||
:%s/\n/,\r/g | :%s/\n/,\r/g | ||
</pre> | </pre> | ||
Then delete the space at the end of the file | |||
OR | |||
<pre> | |||
head -n 10 Kickstarter_2017-10-15T10_20_38_271Z.json | awk '{print "["} BEGIN {RS=""}{ gsub(/\n/,", &") ; print } END{print "]"}' | |||
awk '{print "["} BEGIN {RS=""}{ gsub(/\n/,", \n") ; print } {print "]"}' | |||
</pre> | |||
To explore the category | |||
<pre> | |||
jq '.[].data.category.name' Kickstarter_2017-09-15T22_20_48_432Z.json | sort -u > categorylist09 | |||
jq '.[].data.category.name' Kickstarter_2017-08-15T22_20_51_958Z.json | sort -u > categorylist08 | |||
cat categorylist08 categorylist09 | sort -u > categorylist | |||
</pre> | |||
NB using uniq here won't work as this only looks for the first occurence | |||
[https://webrobots.io/indiegogo-dataset/ Indigogo dataset] | [https://webrobots.io/indiegogo-dataset/ Indigogo dataset] | ||
===Useful file manipulation=== | ===Useful file manipulation=== | ||
jq . filename (for jason queries) | jq . filename (for jason queries) (see also [[Jq json file manipulation]]) | ||
wc -l filename (counts amount of lines) | wc -l filename (counts amount of lines) |
Latest revision as of 10:03, 17 November 2017
Books
AI Playbook by Andreessen Horowitz
An MIT Press book
Ian Goodfellow and Yoshua Bengio and Aaron Courville
Understanding Machine Learning: From Theory to Algorithms
File:Understanding-machine-learning-theory-algorithms.pdf
Presentations
Learn TensorFlow and deep learning, without a Ph.D.
Practice
Script to figure out what an space_action is doing
action = 0 # modify this! o = env.reset() for i in xrange(5): # repeat one action for five times o = env.step(action)[0] IPython.display.display( Image.fromarray( o[:,140:142] # extract your bat ).resize((300, 300)) # bigger image, easy for visualization )
From here
Datasets
in vim the following:
:%s/\n/,\r/g
Then delete the space at the end of the file
OR
head -n 10 Kickstarter_2017-10-15T10_20_38_271Z.json | awk '{print "["} BEGIN {RS=""}{ gsub(/\n/,", &") ; print } END{print "]"}' awk '{print "["} BEGIN {RS=""}{ gsub(/\n/,", \n") ; print } {print "]"}'
To explore the category
jq '.[].data.category.name' Kickstarter_2017-09-15T22_20_48_432Z.json | sort -u > categorylist09 jq '.[].data.category.name' Kickstarter_2017-08-15T22_20_51_958Z.json | sort -u > categorylist08 cat categorylist08 categorylist09 | sort -u > categorylist
NB using uniq here won't work as this only looks for the first occurence
Useful file manipulation
jq . filename (for jason queries) (see also Jq json file manipulation)
wc -l filename (counts amount of lines)
head -n 10 filename > filename2 (sends top 10 lines of code to filename2)
NB. sed can't find and replace newlines as it encapsulates the file being streamed and gets rid of the newlines.